{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "08. Multilayer Perceptrons.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Eldave93/Seizure-Detection-Tutorials/blob/master/08.%20Multilayer%20Perceptrons.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YFzQAyROUw1t",
        "colab_type": "text"
      },
      "source": [
        "# 08. Multilayer Perceptrons\n",
        "\n",
        "by [David Luke Elliott](https://www.lancaster.ac.uk/psychology/about-us/people/david-elliott)\n",
        "/ [GitHub](https://github.com/Eldave93) \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9IYNBQxqtuJq",
        "colab_type": "text"
      },
      "source": [
        "So far we have been focusing on what could be termed \"*shallow*\" machine-learning approaches, and so now we will focus on \"*deep*\" learning. Deep learning is a set of algorithms developed to train many layered artificial neural networks efficiently<sup>2</sup>. Multilayer neural networks are harder to train than the previous simpler algorithms (e.g. logistic regression or support vector machines) as there are typically thousands or even billions of weights that need to be optimized. This means steps need to be taken to ensure the optimization algorithm doesnt become trapped in local minima<sup>2</sup>. \n",
        "\n",
        "The first artificial neural network (ANN) architecture was presented back in 1943 (REF), in a paper describing a computational model of biological neurons. Biological neurons are comprised of a cell body, with a nucleus, with extended branches called dendrites. A long extension called the axon runs from the cell body and has smaller branches at the extremity called telodendria; which each having synaptic terminals. Neurons recive input from electrical signals via synapses, with it firing its own signal if it recieves enough signals from other neurons (action potential). Billions of neurons are arranged in networks to compute conplex computations. Although modern ANN's have moved away from their biological inspiration, the architecture of biological neural networks is still an active area of research<sup>1</sup>. This notebook begins to look at how we can apply artifical networks to classify the outputs of millions/billions of neurons when firing normally (interictal), or synchronously to cause seizures (ictal).\n",
        "\n",
        "The success of modern deep learning methods rests upon the availability of lots of data and GPU hours on modern GPUs to compute matrix-vector computations<sup>3</sup>. Deep learning has accelerated rapidly within the last few years, with one of the main reasons being the \"*democratization*\" of the toolsets used in the field. As can be seen in the table below, starting from 2010 multiple libraries were released for deep learning, with the most popular arguably Theano and TensorFlow. Within the more recent years libraries have become more user-friendly, with Keras quickly becoming one of the dominant go-to deep-learning solutions commonly used by new startups, graduate students, and researchers (such as me)<sup>4</sup>.\n",
        "\n",
        "Open Source Deep Learning Libraries<sup>1</sup>:\n",
        "\n",
        "| Library        | API                  | Platforms                           | Year |\n",
        "|----------------|----------------------|-------------------------------------|------|\n",
        "| Caffe          | Python, C++, Matlab  | Linux, macOS, Windows               | 2013 |\n",
        "| Deeplearning4j | Java, Scala, Clojure | Linux, macOS, Windows, Android      | 2014 |\n",
        "| H20            | Python, R            | Linux, macOS, Windows               | 2014 |\n",
        "| MXNet          | Python, C++, others  | Linux, macOS, Windows, iOS, Android | 2015 |\n",
        "| TensorFlow     | Python, C++          | Linux, macOS, Windows, iOS, Android | 2015 |\n",
        "| Theano         | Python               | Linux, macOS, iOS                   | 2010 |\n",
        "| Torch          | C++, Lua             | Linux, macOS, iOS, Android          | 2002 |\n",
        "\n",
        "![alt text](https://dpzbhybb2pdcj.cloudfront.net/chollet/Figures/03fig02_alt.jpg)\n",
        "\n",
        "So why start to think about deep learning beyond the previously discussed \"*shallow*\" machine-learning approaches? Well firstly they can reduce the need for data preprocessing and feature engineering and can provide simple trainable models built using only five or six different operations. Secondly they are scalable, as they can be easily parallelized on GPUs or TPUs and can be trained by iterating over small batches of data. Thirdly they are reusabile, so they can be trained on additional data without starting from scratch, making them viable for continuous online learning. Furthermore they can be repurposed by taking early already trained layers and applying them to another problem in a similar domain space<sup>4</sup>.\n",
        "\n",
        "**NOTES**\n",
        "- It’s recommended that you run deep-learning code on a modern NVIDIA GPU because some networks (e.g. convolutional networks and recurrent neural networks) are very slow on CPUs.\n",
        "\n",
        "**TODO**\n",
        "- A lot of this work focuses on examples given in the excellent book by Géron (go out and buy it to show support!). I need to update this now to include more of the reading and writing I have done since I wrote this tutorial.\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. Raschka, S. (2016). Python Machine Learning (First). Packt.\n",
        "3. Zheng, A., & Casari, A. (2018). Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists. \" O'Reilly Media, Inc.\".\n",
        "4. Chollet, F. (2017). Deep learning with python. Manning Publications.\n",
        "5. Subasi, A. (2007). Application of adaptive neuro-fuzzy inference system for epileptic seizure detection using wavelet feature extraction. Computers in Biology and Medicine, 37(2), 227–244. https://doi.org/10.1016/j.compbiomed.2005.12.003"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CApjToo3xPOi",
        "colab_type": "code",
        "outputId": "8468eb5c-52c0-425d-dd6c-bfba92c0b59d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "#GPU count and name\n",
        "!nvidia-smi -L"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GPU 0: Tesla K80 (UUID: GPU-9f78268c-06ad-c3da-a65f-62000da5bd4b)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l6ddNsmRxSbL",
        "colab_type": "code",
        "outputId": "fc6cd2c4-f839-4e72-ad73-023ef32c000a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 341
        }
      },
      "source": [
        "#use this command to see GPU activity while doing Deep Learning tasks, for this command 'nvidia-smi' and for above one to work, go to 'Runtime > change runtime type > Hardware Accelerator > GPU'\n",
        "!nvidia-smi"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sat Mar  7 19:22:13 2020       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 440.59       Driver Version: 418.67       CUDA Version: 10.1     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   33C    P8    28W / 149W |      0MiB / 11441MiB |      0%      Default |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                       GPU Memory |\n",
            "|  GPU       PID   Type   Process name                             Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2lZNjsLQSyp5",
        "colab_type": "code",
        "outputId": "a4306cc2-edbe-413e-d4c2-e04bca0172d7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 989
        }
      },
      "source": [
        "!pip install tensorflow-gpu==2.1\n",
        "!pip install hpbandster"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: tensorflow-gpu==2.1 in /usr/local/lib/python3.6/dist-packages (2.1.0)\n",
            "Requirement already satisfied: scipy==1.4.1; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.4.1)\n",
            "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (0.1.8)\n",
            "Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.17.5)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.27.1)\n",
            "Requirement already satisfied: tensorflow-estimator<2.2.0,>=2.1.0rc0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (2.1.0)\n",
            "Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.12.0)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (0.8.1)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (0.9.0)\n",
            "Requirement already satisfied: tensorboard<2.2.0,>=2.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (2.1.1)\n",
            "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (0.2.2)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.11.2)\n",
            "Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (0.34.2)\n",
            "Requirement already satisfied: keras-preprocessing>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.1.0)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (3.1.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.1.0)\n",
            "Requirement already satisfied: protobuf>=3.8.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (3.10.0)\n",
            "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu==2.1) (1.0.8)\n",
            "Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.7.2)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.2.1)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.0.0)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.4.1)\n",
            "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (45.2.0)\n",
            "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (2.21.0)\n",
            "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow-gpu==2.1) (2.8.0)\n",
            "Requirement already satisfied: cachetools<3.2,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.1.1)\n",
            "Requirement already satisfied: rsa<4.1,>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (4.0)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.2.8)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.3.0)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.0.4)\n",
            "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (2.8)\n",
            "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (2019.11.28)\n",
            "Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.6/dist-packages (from rsa<4.1,>=3.1.4->google-auth<2,>=1.6.3->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.4.8)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.1.0)\n",
            "Requirement already satisfied: hpbandster in /usr/local/lib/python3.6/dist-packages (0.7.4)\n",
            "Requirement already satisfied: Pyro4 in /usr/local/lib/python3.6/dist-packages (from hpbandster) (4.79)\n",
            "Requirement already satisfied: ConfigSpace in /usr/local/lib/python3.6/dist-packages (from hpbandster) (0.4.12)\n",
            "Requirement already satisfied: netifaces in /usr/local/lib/python3.6/dist-packages (from hpbandster) (0.10.9)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from hpbandster) (1.4.1)\n",
            "Requirement already satisfied: statsmodels in /usr/local/lib/python3.6/dist-packages (from hpbandster) (0.10.2)\n",
            "Requirement already satisfied: serpent in /usr/local/lib/python3.6/dist-packages (from hpbandster) (1.30.2)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from hpbandster) (1.17.5)\n",
            "Requirement already satisfied: pyparsing in /usr/local/lib/python3.6/dist-packages (from ConfigSpace->hpbandster) (2.4.6)\n",
            "Requirement already satisfied: cython in /usr/local/lib/python3.6/dist-packages (from ConfigSpace->hpbandster) (0.29.15)\n",
            "Requirement already satisfied: pandas>=0.19 in /usr/local/lib/python3.6/dist-packages (from statsmodels->hpbandster) (0.25.3)\n",
            "Requirement already satisfied: patsy>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from statsmodels->hpbandster) (0.5.1)\n",
            "Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.19->statsmodels->hpbandster) (2.6.1)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.19->statsmodels->hpbandster) (2018.9)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from patsy>=0.4.0->statsmodels->hpbandster) (1.12.0)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gk3h1arbH47r",
        "colab_type": "code",
        "outputId": "e0127b73-f604-4f01-e7d0-8c424330c7a6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "import tensorflow\n",
        "tensorflow.__version__"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'2.1.0'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kQT73FewuE6X",
        "colab_type": "text"
      },
      "source": [
        "# Data Preparation\n",
        "\n",
        "First lets start by getting our workspace ready and then loading in the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "esWIWkURtStx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import drive\n",
        "import sys\n",
        "import os\n",
        "\n",
        "CLASS_BACK = '.'\n",
        "\n",
        "# colours for printing outputs\n",
        "class color:\n",
        "   PURPLE = '\\033[95m'\n",
        "   CYAN = '\\033[96m'\n",
        "   DARKCYAN = '\\033[36m'\n",
        "   BLUE = '\\033[94m'\n",
        "   GREEN = '\\033[92m'\n",
        "   YELLOW = '\\033[93m'\n",
        "   RED = '\\033[91m'\n",
        "   BOLD = '\\033[1m'\n",
        "   UNDERLINE = '\\033[4m'\n",
        "   END = '\\033[0m'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9cqvMhRjxqEH",
        "colab_type": "text"
      },
      "source": [
        "## Copy Locally\n",
        "\n",
        "If you have been using Google Colab and data from a mounted google drive, then now is the best time to copy your data to the VM local filesystem to improve training speeds. It means the start of this tutorial will slower as it download BUT training is much faster than if left on google drive<sup>1</sup>\n",
        "\n",
        "In this tutorial we are going to stick with using features familiar to us in previous notebooks. Although deep learning often doesnt require as much data preparation, in its application to EEG data it still occours quite frequently (e.g. REF's).\n",
        "\n",
        "We are moving on to use the TUH dataset now, as deep learning models are much better at handling large amounts of data comparitive to the classical models. Still we are only going to work with a small part of this dataset (most of the patients with absense seizures), so training does not take a long time when going through the examples.\n",
        "\n",
        "**NOTES**\n",
        "- I have a larger dataset of Generalised Epilepsies prepared here (INSERT LINK)\n",
        "\n",
        "---\n",
        "1. https://dev.to/kriyeng/8-tips-for-google-colab-notebooks-to-take-advantage-of-their-free-of-charge-12gb-ram-gpu-be4"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pXw-IPnjxqjs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import os\n",
        "import gdown\n",
        "FILE_PATH = 'TUH_feat.hdf5'\n",
        "\n",
        "if not os.path.exists(FILE_PATH):\n",
        "    gdown.download('https://drive.google.com/uc?id=1-VCuFpjiVEwtgE31te0Z7zdzEKs5HeLU', \n",
        "                './'+FILE_PATH, quiet=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Mqrx8-eu8Yv",
        "colab_type": "text"
      },
      "source": [
        "## Split Data\n",
        "We can see that the data is saved in separate participant ID's"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BCg_Ic98u5GG",
        "colab_type": "code",
        "outputId": "98181cc0-ff5c-4814-e3d2-ea8be61d3616",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        }
      },
      "source": [
        "import h5py\n",
        "\n",
        "f = h5py.File(FILE_PATH, 'r')\n",
        "key_list = list(f.keys())\n",
        "key_list"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['00000675',\n",
              " '00001113',\n",
              " '00001413',\n",
              " '00001984',\n",
              " '00002448',\n",
              " '00002657',\n",
              " '00003053',\n",
              " '00003281',\n",
              " '00003306',\n",
              " '00003635']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_jDQWvwUKcJp",
        "colab_type": "text"
      },
      "source": [
        "Lets split up our dataset into training and test sets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9fU5nIOuJ8Ub",
        "colab_type": "code",
        "outputId": "64ce824e-92de-4d16-9d2f-1cf555e3339e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 245
        }
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "TEST_SIZE = 0.2\n",
        "RANDOM_STATE = 0\n",
        "\n",
        "train_list, test_list = train_test_split(key_list, test_size=TEST_SIZE, \n",
        "                                         random_state=RANDOM_STATE)\n",
        "data_lists = {'train': train_list, \n",
        "              'test': test_list}\n",
        "\n",
        "print(color.BOLD+color.UNDERLINE+'Training List'+color.END)\n",
        "display(train_list)\n",
        "print()\n",
        "print(color.BOLD+color.UNDERLINE+'Test List'+color.END)\n",
        "display(test_list)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[1m\u001b[4mTraining List\u001b[0m\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "['00002448',\n",
              " '00003635',\n",
              " '00001113',\n",
              " '00003053',\n",
              " '00003281',\n",
              " '00001984',\n",
              " '00000675',\n",
              " '00002657']"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\u001b[1m\u001b[4mTest List\u001b[0m\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "['00001413', '00003306']"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RVhiuYZavLXP",
        "colab_type": "text"
      },
      "source": [
        "## Load Data\n",
        "\n",
        "We are going to use the above lists to make our datasets for this tutorial\n",
        "\n",
        "**NOTE**\n",
        "- I have already scaled this data with a standard scaler on each record"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MEb5z1vWvIsN",
        "colab_type": "code",
        "outputId": "e13375ca-1446-4c52-a346-9d7fe8c3e321",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "def make_data(data_lists):\n",
        "  data_dict = {}\n",
        "  for data_type in data_lists:\n",
        "    key_list = data_lists[data_type]\n",
        "    for i, part_id in enumerate(key_list):\n",
        "      if i == 0:\n",
        "        data = pd.read_hdf(FILE_PATH, part_id)\n",
        "      else:\n",
        "        data = pd.concat([data, pd.read_hdf(FILE_PATH, part_id)])\n",
        "    data_dict[data_type] = data\n",
        "  return data_dict\n",
        "    \n",
        "data_dict = make_data(data_lists)\n",
        "\n",
        "X_train = data_dict['train'].values\n",
        "X_test = data_dict['test'].values\n",
        "y_train = data_dict['train'].index.values\n",
        "y_test = data_dict['test'].index.values\n",
        "feat_names = list(data_dict['train'].columns)\n",
        "\n",
        "display(X_train.shape)\n",
        "display(X_test.shape)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "(15566, 1298)"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "(5050, 1298)"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nJPhnk7Mxo8q",
        "colab_type": "text"
      },
      "source": [
        "Change the codes to integers 1, ..., c+1"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1u0mjE3TrSU9",
        "colab_type": "code",
        "outputId": "436d9f89-d3ee-4e57-b4ed-ed3a6978b0b7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "import numpy as np\n",
        "from sklearn import preprocessing\n",
        "display(np.unique(y_train))\n",
        "\n",
        "# lets change integers\n",
        "le = preprocessing.LabelEncoder()\n",
        "y_train = le.fit_transform(y_train)\n",
        "y_test = le.transform(y_test)\n",
        "\n",
        "display(np.unique(y_train))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "array([ 6, 12])"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "array([0, 1])"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DuLXFngAt2SJ",
        "colab_type": "text"
      },
      "source": [
        "# Perceptrons\n",
        "The Perceptron is one of the simplest ANN architectures based on artifical neurons called a threshold logic unit (TLU)/ linear threshold unit (LTU). The inputs and outputs are numerical, with each input associated with a weight. The TLU computes a weighted sum of the inputs, applies a step function, and outputs the result. The most common step function is the Heaviside step function<sup>1</sup>.\n",
        "\n",
        "A single TLU can be used for simple linear binary classification by computing a linear combination of inputs, and if these exceed a threshold, outputs a positive or negative class. Training just means finding the right values of the weights<sup>1</sup>.\n",
        "\n",
        "A Perceptron is a single layer of TLUs with each neuron connected to all the inputs. Special pass through neurons called input neurons, which output whatever input is fed, tend to be first. There is also an extra bias feature commonly added (Bias Neuron), which just outputs 1 all the time<sup>1</sup>.\n",
        "\n",
        "\"*Cells that fire together, wire together*\" Siegrid Löwel\n",
        "\n",
        "This quote is esentially Hebbs rule that the connection weight between two neurons is increased when they have the same output<sup>1</sup>. Perceptrons are trained to have a variant on this rule that takes into account the network error, not reinforcing connections that lead to the wrong output. One training instance is fed at a time, with each making a prediction. For each output neuron that produced a wrong prediction, it reinfroces the connection weights that would have contributed to the correct prediction<sup>1</sup>.\n",
        "\n",
        "Lets have a quick look at how we can make one in sklearn.\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\"."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ErLnhge-v3sG",
        "colab_type": "code",
        "outputId": "cc1261eb-197c-43d5-da42-96f3d2539495",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 93
        }
      },
      "source": [
        "%%time\n",
        "import numpy as np\n",
        "from sklearn.linear_model import Perceptron\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.metrics import make_scorer\n",
        "\n",
        "per_clf = Perceptron(max_iter=100, #tol=-np.infty, \n",
        "                     random_state=RANDOM_STATE)\n",
        "\n",
        "scores = cross_val_score(estimator=per_clf, \n",
        "                         X=X_train, \n",
        "                         y=y_train, \n",
        "                         scoring = 'accuracy',\n",
        "                         cv=5,\n",
        "                         n_jobs=-1)\n",
        "\n",
        "print('CV accuracy scores: %s' % scores)\n",
        "print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores), np.std(scores)))"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CV accuracy scores: [0.98715478 0.98329586 0.96562801 0.97494378 0.97012528]\n",
            "CV accuracy: 0.976 +/- 0.008\n",
            "CPU times: user 109 ms, sys: 162 ms, total: 271 ms\n",
            "Wall time: 2.94 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U-mlNFfuwa4w",
        "colab_type": "text"
      },
      "source": [
        "Perceptrons have a few limitations. They are similar to Logistic Regression classifiers, but Perceptrons do not output a class probability. They only make predictions based on a hard threshold. Also, as was shown by Minsky and Papert (REF), they are incapable of solving some trivial problems. However some of these limitations can be solved by a Multi-Layer Perceptron<sup>2</sup>.\n",
        "\n",
        "Multi-Layer Perceptrons are comprised of a three general layers:\n",
        "\n",
        "1. A passthrough input layer\n",
        "2. One or more layers of TLUs - hidden layers\n",
        "3. A final layer of TLUs - output layer\n",
        "\n",
        "Every layer has a bias neuron and is fully connected to the next layer. When an ANN has two or more hidden layers it is called a deep neural network (DNN). \n",
        "\n",
        "For each training instance the algorithm... \n",
        "- feeds it to the network\n",
        "- computes the output of every neuron in each layer\n",
        "- measures the output error\n",
        "- computes how much each neuron in the last hidden layer contributed to each output neurons error\n",
        "    - this is repeated for each previous hidden later until the algorithm reaches the input layer. This is known as backpropagation (or Gradient Descent using reverse mode autodiff)<sup>3</sup>.\n",
        "- the connection weights are tweeked to reduce error\n",
        "\n",
        "For this to work the step function has to be changed to a logistic function to allow Gradient Descent to make progress each step. As well as the logistic function (output range 0 to 1), back propogation may be used with other activation functions such as the hyperbolic tangent function (tanh), whos output ranges from -1 to 1 helping make each layers output more normalized, and the ReLU function, which is continuous and has an abrupt change in slope<sup>2</sup>.\n",
        "\n",
        "When the output classes are exclusive, the output later is modified by a shared softmax function, where the output of each neuron corresponds to the estimated probability of the coressponding class<sup>2</sup>.\n",
        "\n",
        "As the signal flows from inputs to outputs, this is an example of a feedforward neural network (FNN).\n",
        "\n",
        "---\n",
        "1. Marvin, M., & Seymour, P. (1969). Perceptrons.\n",
        "2. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "3. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985). Learning internal representations by error propagation (No. ICS-8506). California Univ San Diego La Jolla Inst for Cognitive Science."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PNVhqiowxcTP",
        "colab_type": "code",
        "outputId": "47743f75-3a5c-4e60-f27d-df4d7bfe885a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 151
        }
      },
      "source": [
        "%%time\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "\n",
        "per_clf = MLPClassifier(\n",
        "    hidden_layer_sizes = (100,),\n",
        "    max_iter=100,\n",
        "    early_stopping = True, \n",
        "    random_state=RANDOM_STATE)\n",
        "\n",
        "scores = cross_val_score(estimator=per_clf, \n",
        "                         X=X_train, \n",
        "                         y=y_train, \n",
        "                         scoring = 'accuracy',\n",
        "                         cv=5,\n",
        "                         n_jobs=-1)\n",
        "\n",
        "print('CV accuracy scores: %s' % scores)\n",
        "print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores), np.std(scores)))"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/joblib/externals/loky/process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
            "  \"timeout or by a memory leak.\", UserWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "CV accuracy scores: [0.9894027  0.99004176 0.98490202 0.98875683 0.97558625]\n",
            "CV accuracy: 0.986 +/- 0.005\n",
            "CPU times: user 71.2 ms, sys: 79.5 ms, total: 151 ms\n",
            "Wall time: 21.7 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B9HroIJGxUku",
        "colab_type": "text"
      },
      "source": [
        "# Tensorflow (Keras API)\n",
        "\n",
        "TensorFlow is a multiplatform and scalable programming interface for running large-scale machine learning algorithms. It was developed by Google Brain team and was released in November 2015 under an open source license, so now involves many contributions from the open source community. TensorFlow allows execution on CPUs, GPUs, and TPUs; with each in turn having improved performance capabilities<sup>1</sup>.\n",
        "    \n",
        "TensorFlow is built around a computation graph comprised of nodes. Each node represents an operation which may have input or outputs. The values at the edges of the graph are called tensors<sup>1</sup>. Tensors are defined by their 'rank':\n",
        "- rank-0: scalar\n",
        "    - A tensor that contains a number is a scalar. In Numpy this is a float32 or float64.\n",
        "- rank-1: vector\n",
        "    - An array of numbers along one axis is a vector\n",
        "- rank-2: matrix\n",
        "    - An array of vectors is a matrix, which has two axes (rows and columns)\n",
        "- rank-3: 3 dimensional matrix\n",
        "- ect.\n",
        "    \n",
        "So really a tensor is a container for data with an arbitrary number of dimensions. \n",
        "    \n",
        "Tensorflow can break its graph up into chunks run in parallel.\n",
        "\n",
        "In Python there are multipule API's to construct neural networks and it comes with TensorBoard which means you can browse the computational graph<sup>2</sup>.\n",
        "\n",
        "---\n",
        "1. Raschka, S. (2015). Python machine learning. Packt Publishing Ltd.\n",
        "2. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "3. Chollet, F. (2017). Deep learning with python. Manning Publications."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i1VNKLahWcLP",
        "colab_type": "text"
      },
      "source": [
        "**Keras API**\n",
        "\n",
        "Let start to dig into how to start to build a model with the Keras API; the main API since the update to TensorFlow 2.0.\n",
        "\n",
        "Keras is a model-level library, providing high-level building blocks for developing deep-learning models<sup>4</sup>. Keras is modular and has several different backend engines that can be used:\n",
        "- TensorFlow (Recommended)\n",
        "- Theano\n",
        "- Microsoft Cognitive Toolkit \n",
        "\n",
        "TensorFlow has its own Keras implementation called tf.keras, which works well with TensorFlow specific features such as TensorFlow’s Data API.\n",
        "\n",
        "Lets use tf.keras to start to build a similar model structure as before. To do this we need to define the *layers* of the model. A layer processes data by taking one or more tensors as input and outputs one or more tensors. Some layers are stateless, most have a state in the form of the layer’s weights held in one or several tensors<sup>4</sup>.\n",
        "\n",
        "Many different layers and architectures exist. Some of the layers you can include in a deep learning model include:\n",
        "\n",
        "- Fully Connected (Dense) Layers\n",
        "\n",
        "    - A fully connected layer is a set of linear functions of all of the input features used for every independent output. These dense connections allow global pattern detections<sup>2</sup>.\n",
        "    - Usually used with simple vector data, stored in 2D tensors of shape (samples, features)<sup>4</sup>.\n",
        "    \n",
        "    \n",
        "- Convolutional Layers\n",
        "\n",
        "    - Convolutional layers use a subset of inputs for each output. The transformation creates outputs using a few features at a time, meaning they tend to be better for identifying local patterns instead of global ones<sup>2</sup>. \n",
        "    - As the filters move across the inputs in strides, they are commonly used for image recognition, voice recognition or natural language processing<sup>3</sup>. Image data, stored in 4D tensors, is usually processed by 2D convolution layers<sup>4</sup>.\n",
        "    \n",
        "    \n",
        "- Response Normalization Layers\n",
        "\n",
        "    - These layers divide the output by a function of the collective total responce, creating competition for the outputs as it is measured against its neighbours<sup>2</sup>.\n",
        "\n",
        "\n",
        "- Pooling Layers\n",
        "    \n",
        "    - Pooling layers combine multipule inputs into a single output using methods such as averaging, summing, or taking the maximum value. This reduces outputs in intermediate layers to reduce the probability of overfitting the network to the training data<sup>2</sup>.\n",
        "    \n",
        "    \n",
        "- Recurrent Layers\n",
        "\n",
        "    - Recurrent layers have forward and backward connections so that activations can flow back and forth. Neurons in these layers have inputs at time X<sub>(t)</sub> and its own outputs from time X<sub>(t-1)</sub>. This means that the neurons have two sets of weights, one for inputs and one for the outputs of the previous time step<sup>3</sup>. As they have inputs from the previous time steps known as *memory cells*.\n",
        "    - Usually used with sequence data, stored in 3D tensors of shape (samples, timesteps, features)<sup>4</sup>.\n",
        "    \n",
        "\n",
        "Building deep-learning models means creating a pipeline to transform the data. When using Keras, layers you add to your models are dynamically built to match the shape of the incoming layer. Commonly layers are linearly stacked to map a single input to a single output, but other architectures exist (e.g. Two-branch networks, Multihead networks, Inception blocks)<sup>4</sup>. By stacking multipule layers of different types you can create:\n",
        "\n",
        "- Multilayer Perceptrons\n",
        "    - fully connected layers\n",
        "- Convolutional neural networks\n",
        "    - fully connected, convolutional, and pooling layers.\n",
        "- Recurrent neural network \n",
        "    - recurrent, and fully connected layers.\n",
        "    \n",
        "We are going to focus on Multilayer Perceptrons in this tutorial, which is just a sequential stack of Dense layers. We need to specify how many layers and how many hidden units in each layer there are in the model. Having more hidden units for the learning of more-complex representations, but is more computationally expensive and may lead to the model learning unwanted patterns which may not generalise to data that has not been trained on<sup>4</sup>.\n",
        "\n",
        "**NOTES**\n",
        "- [This](http://playground.tensorflow.org) is a great visualisation\n",
        "- Examples of other APIs will be in an extra tutorial I will release\n",
        "\n",
        "---\n",
        "1. Raschka, S., & Mirjalili, V. (2017). Python Machine Learning (Second). Packt Publishing.\n",
        "2. Zheng, A., & Casari, A. (2018). Feature Engineering for Machine Learning principles and techniques for data scientists.\n",
        "3. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "4. Chollet, F. (2017). Deep learning with python. Manning Publications."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A7BZvraJCHKK",
        "colab_type": "code",
        "outputId": "92fc9845-daeb-4519-f61b-ea94f436362b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        }
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "model = Sequential()\n",
        "\n",
        "model.add(Dense(units=300,\n",
        "                input_dim=X_train.shape[1],\n",
        "                activation='tanh'))\n",
        "\n",
        "model.add(Dense(units=100,\n",
        "                activation='tanh'))\n",
        "\n",
        "model.add(Dense(units=1,\n",
        "                activation='sigmoid'))\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 300)               389700    \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 100)               30100     \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1)                 101       \n",
            "=================================================================\n",
            "Total params: 419,901\n",
            "Trainable params: 419,901\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zl_M5TLxxfAO",
        "colab_type": "text"
      },
      "source": [
        "## Activation Functions\n",
        "\n",
        "You may have notice we have had to set an \"Activation\" function.\n",
        "\n",
        "Activation functions convert an input signal to an output signal. Specifically, we get the sum of inputs and weights and apply the Activation function f(x) to get the output of that layer<sup>1</sup>. Without the activation function the model would only be able to learn linear transformations of the input data<sup>2</sup>.\n",
        "\n",
        "First lets look at three common activation functions: Sigmoid, Tanh, and Relu. The sigmoid reduces values into a [0, 1] interval and often used to create a value that can be used as a probability<sup>2</sup>. Relu (rectified linear unit) zeroes out negative values...\n",
        "\n",
        "**NOTES**\n",
        "- code from https://github.com/ageron/handson-ml/blob/master/10_introduction_to_artificial_neural_networks.ipynb\n",
        "\n",
        "**TODO**\n",
        "- add bits in from my writings to improve this\n",
        "\n",
        "---\n",
        "1. https://towardsdatascience.com/activation-functions-and-its-types-which-is-better-a9a5310cc8f\n",
        "2. Chollet, F. (2017). Deep learning with python. Manning Publications."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vxfVpvy_xk-q",
        "colab_type": "code",
        "outputId": "b46ab5cf-5919-49b3-d5e5-dfca09e9e00c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "source": [
        "%matplotlib inline\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "def sigmoid(z):\n",
        "    return 1 / (1 + np.exp(-z))\n",
        "\n",
        "def relu(z):\n",
        "    return np.maximum(0, z)\n",
        "\n",
        "def derivative(f, z, eps=0.000001):\n",
        "    return (f(z + eps) - f(z - eps))/(2 * eps)\n",
        "\n",
        "z = np.linspace(-5, 5, 200)\n",
        "\n",
        "plt.figure()\n",
        "\n",
        "plt.plot(z, np.sign(z), \"r-\", linewidth=1, label=\"Step\")\n",
        "plt.plot(z, sigmoid(z), \"g--\", linewidth=2, label=\"Sigmoid\")\n",
        "plt.plot(z, np.tanh(z), \"b-\", linewidth=2, label=\"Tanh\")\n",
        "plt.plot(z, relu(z), \"m-.\", linewidth=2, label=\"ReLU\")\n",
        "plt.grid(True)\n",
        "plt.legend(loc=\"center right\", fontsize=14)\n",
        "plt.title(\"Activation functions\", fontsize=14)\n",
        "plt.axis([-5, 5, -1.2, 1.2])\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEJCAYAAACXCJy4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3wU1drA8d9JbyQEAgFCCR0REAQE\nRSWKWNALKvCKqBRFUBQ7gsoF9HpRsaKoF0RBxGvHK80uUar0pnQIkEJCAqSQQnb3vH/MJtmEtE02\n2d3s8+Uzn52dOTPz7GR5cnLOzByltUYIIUTd5+XsAIQQQtQOSfhCCOEhJOELIYSHkIQvhBAeQhK+\nEEJ4CEn4QgjhISThC6dSSsUppZ6qhePMVErtqYXjeCml5iml0pRSWikVU9PHrCCeRUqpFc6MQbgO\nJdfhi8pSSl0KbAY2aq372bntTGCY1rpLieWNgHNa62wHxRgNHAV6a6232CwPAfy11mmOOE45x78F\nWArEAEeA01rr8zV5TOtxY4DVQCOtdarN8jCM/+dnazoG4fqkhi/sMQ54D+iilLrIETvUWp9yVLKv\n4DhZNZ3srdoBSVrr9Vrrk7WR7MujtU6XZC8KSMIXlaKUCgRGAvOBr4H7SinTTCn1qbU5I1sptUMp\ndY1SagwwA7jY2syhrcuKNekopf6rlPqmxD69lFInlFJPWN/fqJRao5Q6o5Q6rZT6scQvn6PW183W\n48RatyvWpGPd7z+t+85TSu1WSg2xWR9t3X6oUupn6+f5Wyk1sJxztAh4E2hp3TbOujxWKTW3ZFnb\nphZrmfeUUrOUUqlKqRSl1GtKKS+bMn7W9cesMR9RSj1i/atmtbXYKeuxF5VxHH+l1FtKqWSlVK5S\naqNS6kqb9THW7Qcopf60fu4t1r/uCsqEKaU+scaYa43jsbLOi3AdkvBFZQ0DjmmtdwOfAKOUUr4F\nK5VSwcDvQDRwK9AVeMG6+gvgdWA/0NQ6fVHKMZYAN1ubIQr0t5b/zPo+GHgLuAyj2SQdWK6U8rOu\nv8z6eqN1u9vL+DyPApOBKdZYvwWWKqW6lyj3b+Bt4BKM5qzPrc1DZe3zBSDeeuzeZZQry12ACbgC\neBh4DLjDZv3HwCjgCeAijF+6Z4ETwFBrmYutx360jGPMtu7zXqAHsBv4QSnVtES5l4CpwKVAGvCp\nUkpZ172Icc5uATpa95Vg52cVzqC1lkmmCicgFnjKOq+AOIw2+YL19wOZQEQZ288E9pSyPM5mvz5A\nMnCfzfoFwE/lxBUMmIErre+jAQ30Ku/4GAlqeimfcUmJ/UywWR9lXXZlOfE8BcSVst+5JZYtAlaU\nKLOhRJmfgQXW+fbWY99YxnFjrOsjyjqO9VydB0bZrPcGDgMvltjPDTZl+lmXNbe+XwZ85OzvpEz2\nT1LDFxVSSrUDrgT+C6CN//WfUrxZpwewS9t0GNpLa23CqPnfZT2uP0bNdYlNLG2tTT+HlVIZGL8g\nvICWdnyeUKAZsK7EqrVA5xLLdtnMJ1pfG1f2WHbaVeJ9os2xegAWippuqqIt4IvN59Zam4EN2Pe5\n3wfuUErttDY79a9GTKIW+Tg7AOEWxmHUBI8X/VWPAlBKtdBan3DgsZYAG5RSUUAfwA/jqpcCKzCa\nTCZg1NJNwN/Wco5Q8rK1/MIVWmvr57e3omTBer5s+JZSLr/Ee12FY1VVmZ/bZp0XgNb6e6VUK+Am\nYACwUin1ldZ6bM2HKapDaviiXEopH2A08AzQ3Wa6BKMWWPCffDvQTSkVUcauzmP80iiX1noTcAi4\nE6Om/53WOssaS0OgEzBLa/2L1novUI/iFZeCq2LKPJbWOgOj1lry0tIrMX55ONopjHZ1W5fYuY8d\nGP9fryljfYWfG6Pp5jw2n1sp5Q1cjp2fW2udqrX+RGs9BuMvvdHWv8iEC5MavqjIzUAE8IEucVmj\nUupz4AGl1L8wmnumAt8ppaZi1L67AJla69UYbfWtrFd7HLcuzyvjmJ9i/FURTfFO1zNAKnC/UuoE\nRpv6qxi1/AIpQA5wg/UqmVytdXopx3gVeEEpdRDYCtwNXIXRSelovwFvKaUGY3RcTwBaYJyTStFa\nH1BKfQksUEo9CmwDmgPRWutPgGMYNfGblVLLgZyCX5Q2+zinlHofeEUplYpxRdPjQCTG5baVopR6\nwXr8vzByyO3AkXJ+nsJFSA1fVOQ+YHXJZG/1FUZSHqi1PodxRU08sBzYAzxPUXPAN8Aq4FeMGu+d\n5RxzCcbVH+nATwULtdYWjCtMuln3/y7wTyDPpowJeATjF0Yi8F0Zx3gbI+nPtu7rNmCo1npnOXFV\n1Uc20zqMzu1vq7CfURi/WN8G9mF0yIYBaK0TMC59/TdGv8bc0nfBFIx+koUYfzV0w+gITrIjjjzr\ncXZifJ56wD/s+yjCGeROWyGE8BBSwxdCCA8hCV8IITyEJHwhhPAQkvCFEMJDuOxlmRERETo6OtrZ\nYXDu3DmCg4OdHYZLkHNRxFXPhc7XZO3KAgUh3UJQPiXv93I8Vz0XzuAK52Lr1q2pWutGpa1z2YQf\nHR3Nli1bKi5Yw2JjY4mJiXF2GC5BzkURVz0Xx185zpFdR4i4PYIuX3epeAMHcNVz4QyucC6UUsfK\nWidNOkLUEVprTi4+CUCT0U2cHI1wRZLwhagjMrdmkv13Nr6NfGlwYwNnhyNckCR8IeqI5MXJADQe\n2RgvX/mvLS4k3woh6gDLeQvJ/zUSvjTniLJIwheiDjj7x1lMaSaCuwQT0r2sAbmEp3PZq3SEEJXX\n4LoG9NrVi/y0fGzGLBCiGEn4QtQRIV2lZi/KJ006Qrg5U5ap4kJCIAlfCLe389qdbO2zlexD2c4O\nRbg4adIRwo2dTzlP9r5sUOAfJSMMivJJwhfCjfk19uOKk1dwbvc5vAMrHDJYeDhp0hHCzXkHeRPa\nJ9TZYQg3IAlfCDeVdzIPc7bZ2WEIN+KQhK+U+kgplaKU2lPGeqWUelspdUgptUspdakjjiuEJzsy\n+Qjrm6wndVmqs0MRbsJRNfxFwI3lrL8JaG+dxgPvO+i4QngkU6aJU0tPYc40E9xVnkUvKschnbZa\n6z+UUtHlFBkCLNZaa2CjUqq+Uqqp1jrJEccXotZZLJCf77TDn/oiGUu2hbCrQgls7uPUWJTJ5LTj\nWyyQm1s05edDQTgmk3Uyq8L5YstLTGYzaG3sU2Mzrwvmlc08pc4fPNiMLX+ayy1TwHa+5PuqrqtI\nbV2lEwWcsHkfb11WLOErpcZj/AVAZGQksbGxtRRe2bKyslwiDlcg56JI10cfRf/9N9pJjzE4aX4N\n6E7jtdOxBP7glBgKXAVY7NwmSweTTCTJRJJKQ9J1GJmEkk6YdbLOW5fnEEAOgeQQSK51PpcA8gio\niY9UDR2cHUC5XOqyTK31fGA+QK9evbSzR44B1xjBxlXIuSiSlZ2N2r4d1a1brR87Jy6H9NZ/4hXg\nRWTy//AKde5/45Lfi4wMOHoU4uKKXo8fh5MnITnZmM6dc9zxAwKKJj8/8PEpmnx9i78va7m3tzEp\nBV5eYNFmzpkyOG/O47zFZjLnkW/Jo3vTS6jnH4KXF2xN2sLB0wfIt+SRa8rBjAmTzgM0kSGRjL10\nNEa9QPPS2n+Dsq2SF80Pan8zvaJ6ArA1aSsrDywv/kFttpsZMwOljBb5eVvnkZgZX1Ruddnnqra+\nKQlAC5v3za3LhHBLymw2MoMTJH9iPAY54rYIfJyY7HNyYNcuWLWqCcuWwZ49xpRUiYbagACIjDSm\niAgICyt7Cg2FoCAIDDS2Cwwsmvf3t+/HsObYGg6dPkTyuWSSs5KNV+v8oPaDmD1wNgA7Tu6mx7we\nZe7nP/eu44oWVwDwyPeL2bDpnQvKhPiF0KlpT14aM9q6RPHn4tV4K28CfAII9A00Xn2M1+Gdc+nX\n0ii5LzWYIcei8Pfxx8/bD18vX/y8/Yx5b19iosHL+sfl3WcGYrKYCss1D3uxzLhr69uyDHhYKfU5\n0AdIl/Z74da0dkrC11oXDnRS28+9T0qC9euNad062LatoOm+U7FyAQHQujVERxe9tmoFTZsWJfnQ\nUHBUa1h6bjq7kndxIuMEx9OPcyL9RNF8xgmOP3acYD+jY/ufq//J78d+L3U/nSKKPkdEUASdIjrR\nILAB4QHhhAeGG68B4YQFhNEyrGVh2cf7Ps7oS0ZTz78eu7fs5vqY6wn2C8ZLXfj9+HXUr5X6TJ0i\nOhWLpzxtwttUqhw4KOErpT4DYoAIpVQ8MAPwBdBa/wdYBQwCDgHZwFhHHFcIZ1FaG20AtSxjQwY5\nh3Lwa+pH+HXhNXqs8+dh7VpYtcqY9u4tvl4p6NIFIiOTGTAgkosvNt5HRzv2d2G+OZ+jZ49yMO0g\nB08f5GDaQa5tfS1DOw8FIDYullu/uLXM7VPOpdDarzUA17e9npZhLYkMjiQyJLLYa9N6TQu3aR7a\nnL0P7S1rl8W0Dm9N63Bj/4n+idTzr1fVj1rjHHWVzp0VrNfAQ444lhCuwFlNOic/NgYpj7w7EuXt\n+A5jsxlWr4bPPoNvvoH09KJ1ISHQty/06wdXXGHMh4ZCbOxeYmIiq33sfHM+vt6+he/H/G8Ma4+v\nJe5sHGZd/AYzszYXJvy2DdrSJ6oPLcNa0jKsJS1CWxivYS1oEdqCRsGNCrd79qpnqx2nO3OpTlsh\n3IYTmnTMuWZSvkgBIHJU9ROsrRMn4D//gY8+MjpXC1x8Mdx8MwwaZCR5X9+y92GPjLwMtiZuZXPi\nZraf3M7u5N0cPnOY00+fJtA3EID4jHgOnzmMQtEqrBXtG7anfQNjurzF5YX76tK4CxvHbXRMYHWc\nJHwhqkBZLLXepJO2LA1zupmQS0MI6eKYwU7WroU334T//c+4VhygXTsYORLuvBM6Va4ZudI2xm/k\n3u/uZV/qPjTFLyBXKA6fOUyXxl0AeOOGN/Dx8qFNeBsCfFzt8kv3JAlfiKqwWGq9hn/2j7MANBlV\n/c7a9ethxgz45RfjvY8P/N//wUMPGU021elQzTPlsTlxM7/H/c7vx36nR5MevDLwFQAaBTVib+pe\nfL18uaTJJfRu1pueTXvSLbIbFze+mCDfoML9dIus/Ute6zpJ+EJUgTM6bdu/056m45oS0LLqtd19\n++Dxx+EH671aoaHwyCMwcaJxFU1V7Ty5k+/2f0dsXCwb4jeQa8otXJeUlVSY8NuEt+HPcX9ySeQl\n+PvI8/trmyR8IarAGZ22Sinqda/aFSDnzsGLL8LrrxuXUoaEwGOPGcm/QQP795eWnYaft1/hFSkf\nbf+Itze9Xbj+4kYX079Vf/pH9+fqVlcX+wyXRV1Wpc8gqk8SvhBVUcudtjlxOQRGB1Zp2x9/hPvv\nNzpmwZifNcu44cke+1L38c3f37Dy4Er+TPiTebfMY9yl4wAY1nkYJouJAW0GcFXLq4pdGSNchyR8\nIaqgNjtts3ZmsaX7Fhre0pCuy7tWeru8PHjmGaNTFuDSS+G996BPn8ofe0/KHr7++2u+/vtr/jr1\nV+FyXy9f4jOKbue/qtVVXNXqqsrvWDiFJHwhqqIWO23P7T2HV7AX/i0q3+a9b59xlc2OHcbvpRde\ngClT7P8dNen7ScTGxQJQP6A+t3a6lVs73sqANgMI8XPMlUKi9kjCF6IKVC026USOiKThLQ2xZFfu\nmZQ//GBccZOZCW3awH//W3GtPj03na///prFuxbzxvVv0LOZ8RCvsd3H0i68HcM6D+Oa1tfg5+1X\n3Y8jnEgSvhBVUcvX4fuE+EAlKtTvvQeTJhnhDR8OCxYYV+KURmvNxviNzN82ny/2fEGOKQeAJbuW\nFCb8UZeMYtQloxz1MYSTScIXogpULTXppG9IJ6RHCN4B5f9yMZvhySdhzhzj/T//CTNnlh3iwu0L\neXPjm+xO2V24LCY6hlHdRhU+skDUPZLwhaiC2ui0zT+bz45rduAd7E3fo33LfBSy2QzjxsGiRcaj\nDxYsgFEVVMo3JWxid8puGgU1Ymz3sdzf837aNWjn+A8hXIokfCGqohZq+Ke+PIXO04RcGVJmsrdY\njMssFy0ynhm/YgVcc03Req01q+NW8+bGN7mr612M6DICgEf7PkpMdAy3XXSbtMt7EEn4QlRBbXTa\nnlxsPMWsrEcpWCxGzX7hQiPZr1oF/fsb68wWM1/9/RUvr32Znck7ATibe7Yw4dvzvHVRd0jCF6IK\narpJJ/tQNhnrMvAK9iLi9gvvkNLaeCRCQbJfudJI9ufN51myawkvr32Zg6cPAhAZHMlDvR/igV4P\n1Fi8wj1IwhfCXgWPlazBAcwLRrVqNKyRcYVOCXPmwLvvGkP8rVgBBUPKfrT9Ix5c+SBgPLdmSr8p\njL5ktDy3RgCS8IWwn8WC9vKiptK9tujCcWtLa85ZtgyeeMKY/2ihmUYX7wWMRwrf0+0ePtvzGfdf\nej8juozAx0v+i4si8m0Qwl41nPDT16STG5eLfwt/6sfUL7Zu2zbjDlqt4f8e3s3zacNIXZTK0UeP\nEuofSrBfML+PKX3MViFqf4w2Idyd2VyjzTmFwxjeE4nyKjrOqVMweLAmOxvC+y7ny4bdOJB2gPCA\ncI6eOVpj8Yi6Q2r4QtjLYkHXUIetOdvMqa9OAcWbcywWuG1EOgkJYdBiHWeuG0ZUaBTT+09nbPex\nxcaCFaIskvCFsJfZjK6hGn7qt6mYs8yE9g0lqGPR6E9vvKFZ91sYBKZR/+4HmXbTLCb2nlg4/qsQ\nlSEJXwh71eBNV4XNOaMiyTXlkp2fzeE9DXjmGeMXzK1TlrFoyhrCAsJq5PiibpM2fCHsZe20dfhu\n8yycTzqP8lNsuGQDneZ24qGlzzJiBJhMxnX3384YK8leVJnU8IWwVw112nr5exH8azAvffwS3/78\nLQCZ3wzk9BHo0QNmz3b4IYWHkRq+EPaqgU7b7Pxsnv31WbrP68632d8SERTBE02WcXrtUPz8YMkS\n4yYrIapDavhC2MvBnbZZ57PoP6s/iWcTyQ/PZ/yl45ne7yVi+hijiz/3HHTu7LDDCQ8mCV8Iezm4\n0zbEL4TxW8fTcUVHfGb6cOU/ruSZZ+DQIbj4Ypg61WGHEh5OmnSEsFc1O20t2sLbf77Nb0d/K1x2\nVfRVeAV6ccngS9i+HV591egmWLAA/OTpxcJBpIYvhL2q0Wl76PQh7v3uXtYcX0OHhh3Y8+AefL19\n6fxOZ0wvmVCB3jxwhXGIRx6Bvn0dHLvwaJLwhbBXFWr4BbX6Z399lhxTDpHBkbxy3SvF7pD1CfFh\nyRLYtAmaNoUXX3R04MLTScIXwl52JvwDaQe497t7WXdiHQB3d7ubOTfOoUFgA/LT8jkbe5aGtzQk\n1+zFM88Y28yaBfXq1UTwwpNJwhfCXmZzpTtt8835DPxkIMfTj9MkpAnzbpnH4I6DC9enfJ7CwYcP\n0mhYI77pejHx8XDppRWPSStEVUjCF8JedtTwfb19eW3gayw7sKywVm+r4FEK3tdE8MpkY9mbb9b4\n6InCQ0nCF8Je5XTami1m5vw5h3xzPlOunALA8IuHM/zi4ReUPbf3HJmbM/EO9ebV9RFkZ8Ptt8PV\nV9do9MKDScIXwl5l1PD3p+5n7Hdj2RC/AV8vX+7seictw1qWuZuCYQx9rmvMR5964+cnj08QNUv+\ncBTCXiUSvtli5vX1r9N9Xnc2xG+gWb1mLL1jabnJXps1Jz8xmnM+PRUJwAMPQNu2NRu68GxSwxfC\nXjZNOvtS9zH2u7FsjN8IwJjuY3jj+jcIDwwvdxdnVp/hfMJ5VFQA764JIzCQwit0hKgpDqnhK6Vu\nVErtV0odUkpdcCO4UmqMUuqUUmqHdRrniOMK4RQ2NfzJP09mY/xGoupFsXLkShYOWVhhsgdI/tho\nzlkX3ARQTJwITS4cr1wIh6p2DV8p5Q28CwwE4oHNSqllWuu/SxT9Qmv9cHWPJ4SzaZvLMufeNJfm\n9Zrz0nUvUT+gfgVbGkyZJk4tNYYxnHsgkqAgePrpGgtXiEKOqOFfBhzSWh/RWp8HPgeGOGC/QrgU\ns8XM7HWzGbT1cczWwcVb1W/F+7e8X+lkD3Dqm1NYsi2cCA8jiUAmTYLGjWsqaiGKOKINPwo4YfM+\nHuhTSrmhSqmrgQPA41rrEyULKKXGA+MBIiMjiY2NdUB41ZOVleUScbgCTz4Xx84d45X9r7A3cy8A\nsY1aEl7VczHHePn8TBMCA01cfvmfxMbmOyZQJ/Dk70VJrn4uaqvTdjnwmdY6Tyk1AfgYuLZkIa31\nfGA+QK9evXRMTEwthVe22NhYXCEOV+CJ58JkMfH6+teZsX0GeeY8moc254NWk+i79FPqV+Fc5B7L\nZeOOjZi8vPjd0ojHH/dhyJB+jg+8ijIyMkhJSSE/v/K/gMLCwggICKjBqNxHTZ8LX19fGjduTGho\naJW2d0TCTwBa2Lxvbl1WSGudZvN2ASBXGwuX9/epvxnzvzFsTtwMwLge43jt+tcIW7+VM16fVWmf\nuSdy8Y4O5Ne4epgDfHj0UUdGXD0ZGRkkJycTFRVFYGAgqpJPBM3MzKSePPgHqNlzobUmJyeHhAQj\nvVYl6TuiDX8z0F4p1Vop5QeMAJbZFlBKNbV5OxjY64DjClGjvtv3HZsTN9M8tDk/3PUDHwz+wBhA\nvBoDoNS/sj4LYy7jdTowdqxrtd2npKQQFRVFUFBQpZO9qD1KKYKCgoiKiiIlJaVK+6h2DV9rbVJK\nPQz8CHgDH2mt/1JKvQBs0VovAx5RSg0GTMBpYEx1jytETTh3/hzBfsEATO43GbM2M+mySUaiL2A2\nV3kAlIQEWPKpwuzlwxNPOCJix8nPzycwMNDZYYgKBAYG2tXkZsshbfha61XAqhLLptvMPwPIbSXC\nZZ03n2fWmln8Z8t/2PHADpqENMHHy4dpV0+7sHAVR7xKW5XGvO+Cyc8PYPhwaNfOAYE7mNTsXV91\nfkbyaAXh8bYkbqHX/F48//vzJJ9LZuWBleVvYLHYPeKVOdfM3yP3EjN/I03IYfLkagQsRBXJoxWE\nx8rJz2Fm7Exe2/AaFm2hbXhbPhz8If2j+5e/YRWadExnTaRENyBxZy6dYgLp3bsagQtRRVLDFx5p\nU8Imus/rzuz1s9Fa80TfJ9j14K6Kkz1UqdPWq6E/j6Z25lF6SO2+Bpw6dYqJEycSHR2Nv78/kZGR\nDBgwgJ9//hmA6OhoXnvtNSdH6XxSwxceyWQxcTDtIBdFXMRHQz6ib3M7RguvQg3/f/8zOmw7dlTc\ndJOdwYoKDR06lOzsbD788EPatWtHSkoKv//+O2lpaRVv7EGkhi88xuaEzYXzV7S4ghUjV7B9wnb7\nkj3Y3YaftjKNn2em4o2Fhx+2u/lfVODs2bOsWbOGl19+mQEDBtCqVSt69+7NU089xYgRI4iJieHY\nsWNMnjwZpVSxTs/169fTv3//wssdH3zwQTIyMgrXx8TE8MADD/Doo48SHh5OeHg4kydPxmKxOOOj\nVpskfFHnHU8/zpDPh3DZgsv46fBPhcsHtR+Ev4+//Tu08yqdv588ysi/9zAgII3Ro+0/nChfSEgI\nISEhLFu2jNzc3AvWL126lObNmzN9+nSSkpJISkoCYPfu3Vx//fUMHjyYnTt3snTpUnbs2MG9995b\nbPtPP/0Ui8XChg0bmDdvHvPnz+ett96qlc/maNKkI+osk8XEnI1zmBE7g3P556jnV4/U7NTq79iO\nJp2sXVmY92eRgQ8X3dsQuSHV8Xx8fFi0aBH3338/8+fPp0ePHvTr14/hw4fTp08fGjRogLe3N/Xq\n1aOJzTOoX331Ve644w6efPLJwmXvv/8+PXr0ICUlhcbWu+KaNm3K22+/jVKKTp06ceDAAd544w2e\ncLUbKSpBaviiTvoz/k96f9Cbp35+inP55xjeeTj7Ht7HyK4jq79zOzpt4+Ybo1qtpjEPPuKm/92U\nKneqFxpaYRm7JzsNHTqUxMREli9fzk033cT69evp27cvs2bNKnObrVu3smTJksK/EEJCQujXz3iu\n0eHDhwvL9e3bt1gz0OWXX05CQkKxph93ITV8Ued8vudz7vzmTgCi60fz7qB3GdR+kOMOYDajK5GU\nLCYLSR8n4wucvSySjh0dF0Kt0rrc1a7yLJ2AgAAGDhzIwIEDmT59OuPGjWPmzJk89dRTpZa3WCyM\nGzeOxx9//IJ1UVFRNR2uU0jCF3XOTe1uomVYS+7qehfTrp5GkG+QYw9QyRp+2g9n8M3K5ziBDJ1W\ntacbiqrr3LkzJpOJ3Nxc/Pz8MJvNxdZfeuml/PXXX7Sr4JbnP//8E611YS1/48aNNGvWrMpPrHQm\nN/0bU4giKw+sZOAnA8nOzwYgLCCM/Q/vZ9aAWY5P9lDpTtsds43mnM3hTbhpkFyaU1PS0tK49tpr\nWbJkCbt27eLo0aN89dVXzJ49mwEDBhAaGkp0dDRr1qwhISGB1FSjH2fKlCls2rSJBx54gO3bt3Po\n0CFWrFjBhAkTiu0/MTGRxx57jP379/P111/z6quvlvpXgTuQGr5wW3tP7eXJn57k+0PfAzB/63we\n6/sYAAE+Nfh89kp02uafzUetS8UCtJ0Qibd3zYXj6UJCQujbty9z5szh0KFD5OXlERUVxciRI5k2\nzXgW0gsvvMCECRNo27YteXl5aK3p1q0bf/zxB9OmTaN///6YzWbatGnDbbfdVmz/d911F2azmT59\n+qCU4r777pOEL0Rtic+IZ2bsTBbuWIhFWwj1D2Vm/5k81Puh2gmgEtfhH5h/Ch+LZruqz12Py+Ag\nNcnf359Zs2aV20Hbt29fdu7cecHyXr168cMPP5S7fx8fH+bOncvcuXOrHauzScIXbuWdP9/h6V+e\nJteUi7fyZkLPCbxwzQs0Dq7FB8tXoknn0LsnCQNO92riUs+8F55NEr5wK1GhUeSachneeTgvXvsi\nHRp2qP0gzOZyO22zDuQQdjyDHLzo/8+IWgxMiPJJwhcuK9eUy4fbPuRU9ilmxswE4LZOt7HzgZ10\ni+zmvMAqqOFvmHkSX2BbSCOeuVn+i7kzVx6QvCrk2yhcTk5+DvO3zueVda+QlJWEt/Jm1CWjaBPe\nBqWUc5M9VNhpu+YvPzoSQEAvkk0AACAASURBVPiIJlUdCVGIGiEJX7iMzLxMPtj2AbPXzSb5XDIA\nl0RewvT+04muH+3c4GyV02mbkACz/opCeTfj+Au1HJcQFZCEL1xCRl4Gree05nTOaQB6Nu3J9P7T\n+UeHf7jesHvlNOksXGg08Q8dqmjatJbjEqICkvCF0+xP3U+Hhh1QShHqH0pMdAwp51KY2m8qg9oP\ncr1EX6CMTlvTOTMn30wggkjGjavCUziFqGGS8EWtyjfns2z/Mt7b8h6/Hf2Nn+/5mevaXAfAktuW\nEOgb6OQIK8FiKfVZOutfSWXY6SNc4pvKwIGXOiEwIconCV/UisTMRD7Y+gHzt80nMTMRgCDfIA6d\nPlSY8N0i2YOR8Eu5dXbVjgACaETEdeFyZ61wSZLwRY17aOVDzNs6D7M2Hl51UcRFTOw9kXu63UNY\nQJiTo6sCs/mCTttz5+C92DAyCWPvG06KS5QpJiaGLl26uMTdspWJpUuXLgwbNoyZM2c69NiS8IVD\naa3ZmbyT5qHNiQgybjoqeB3WeRgTe00kJjrGddvnK6OUTttvv4XMTOjTBzp1clJcHuzUqVPMmDGD\nVatWkZSURP369enSpQtTp05l4MCBLF26FF9fX2eHCeDUWCThC4fYe2ovX/z1BV/+9SV7U/cy+7rZ\nTO43GYBJfSYxsfdEIkMinRylg5TotNVaE//sYfoSzuhRDQA3/mXmpioaxLxBgwZOjrCIM2OR20JE\nle1L3ce///g33d7vRuf3OvP878+zN3UvDQMboikaNCMiKKLuJHu4oNP20MpM+p6IZwr7+L9h5Q8W\nIhyvokHMwWhGefjhhwu3SU5OZvDgwQQGBtKqVSsWLlxIly5dijWhKKV4//33GTJkCEFBQXTo0IHV\nq1cTHx/PDTfcQHBwMN27d2fbtm3F4lm6dCldu3bF39+fFi1a8O9//xttM4hMyVhSUlIYMmRIYSwf\nffRRDZ0pSfjCDvmWfPLN+YXvp6+ezrTV09idspv6AfUZ230sP9z1A0lPJvF0v6edGGkNK9Fpu/nf\nxk1iJ9o3pkFj+S9V2yoaxLw0o0eP5tixY/z222989913LFmyhGPHjl1Q7sUXX2TEiBHs3LmTXr16\nMWLECO677z4mTpzI9u3badasGWPGjCksv337doYPH87tt9/O7t27efnll3nppZfKba8fM2YMhw4d\n4pdffuF///sfixcvJi4uzt7TUCnSpCPKdSL9BL8e/ZUVB1bw/YHv+aL5F9zS4RYARnQZQZBvEMM6\nD+P6ttfj5+3n5GhriU2nrTnPQr1NRsLv8EiT8rZyW+r5spuo5t0yjzs7GMNJzt86nwkrJpRZVs8o\nquX2nN+TbUnbKixXGRUNYl7S/v37+fHHH9mwYQN9+/YFYNGiRURHR19QdtSoUdx5p/H5nn32WT77\n7DNuuOEGhgwZAsDTTz/NNddcQ2pqKhEREcydO5f+/fvz/PPPA9ChQwcOHjzIK6+8wqRJky7Y/4ED\nB/j+++9Zu3Zt4Xi6H3/8MW3atLHrHFSWVEfEBb7Y8wXjl4+n/TvtaflWS8Z+N5Zv9n5DtjmbLYlb\nCsvdftHtLLp1Ebd0uMVzkj0U67Rd/+Zp6llMnPAJYsADIU4OzHPZM4j5vn378PLyolevXoXLWrRo\nQbNmzS4o261b0XObIiONZsmuXbtesCwlJQUwEnhB4i5w5ZVXljno+d69e/Hy8uKyyy4rXNaqVatS\nY3EEqeF7MIu2cDDtIJsTN3NX17sKr5yZtXYWu5J3ARDqH8rVra7mxrY30vB0Q0bEjHBmyK7BZkzb\no/85SUvg3JVN8PGpm521FdW4MzMzARjfczzje46v1D63jt9a7bhKsncQ88qwvZqm4P9HacssFkuF\n+yrvyrTaumpNEr4HSTmXwrakbWyM38jG+I1sStjEmdwzAFze/HLaNmgLwEO9H+JMzhmubX0tPZr2\nwMfL+JrUtUfFVpnZjFaK9BP5ND2Whhm4cmYd6pSuI2wHMbfVqVMnLBYLW7duLWzyiY+PJzExsdrH\n7NChA+vWrSu2bO3atTRv3px69epdUL4glk2bNnHFFVcAcPz4cYfEUhpJ+HWQyWLiQNoBzpvP071J\ndwB2nNxBj3k9LijbNKQpl7e4nDxzXuGyytbSPJbFgvbxYfU/U6iPZn9YOAP6y7NznCUtLY3hw4dz\n77330q1bN+rVq8eWLVuKDWJuq2PHjtxwww088MADvP/++wQEBDB58mSCgoKqXdOeNGkSMTExzJw5\nk5EjR7J582Zef/31Modf7NixIzfeeCMTJkxg/vz5BAYG8sQTTxAYWDN3nUvCd3P7UvexNXEr+1L3\nsT9tP/tS93Eg7QB55jyua3MdP9/zMwCdIjoRHhBO50ad6du8L32b96VPVB+ahzZ375ugnMFsBj8/\nzn17kvpAyO11s7PWXVRmEPOSCjp5Y2JiaNy4MS+88AJHjhwhIKB64w93796dr776ihkzZjBr1iwi\nIyOZOnVqscswy4rl2muvJSIighkzZhT2CTiasr0+1JX06tVLb9mypeKCNSw2NpaYmBinHPu8+TzH\n048TdzaOo2eOEnc2jrj0OGb0n1E4tN+E5ROYv23+BdtG14/mmuhr+GhI0TW9WutqJXdnnguXMn48\nsWfaw9e9OYc3fROuoGEz9394zt69e7nooovs3i4zM7PU5gp3kpqaSrNmzfjss88YOnRolfdTW+ei\nvJ+VUmqr1rpXaeukhu8EWmvSctJIykwiMTORxMxE6gfU57aLbgMgKTOJyxZcRkJGQrEbmAoM7zy8\nMOFf1eoqTueeplPDTnSM6EiniE50aNiBUP/QC7aTmryDWCwc2NidDsDx1o24uQ4ke0/z22+/kZmZ\nSdeuXUlJSeG5554jIiKCG2+80dmh1ShJ+NVkspg4de4U6XnpZORlcDrnNKnZqYXTo30epVFwIwCe\n+PEJ/rv7v6TlpGGymIrt56qWVxUm/PDAcOIz4vFSXrQIbUHr+q2Jrh9NdP1oWtdvTc+mPQu3u7vb\n3dzd7e7a+8ACy3kLDRMCARNtHpTmHHeUn5/PtGnTOHLkCEFBQfTt25c//viD4OBgZ4dWoxyS8JVS\nNwJzAG9ggdb65RLr/YHFQE8gDbhDax3niGPbIyc/h7ScNLLzs0ud7rj4jsJa8Lwt8zhy5gj74/az\n4PSCwoSenpvO4I6DeeEaY/y6bUnb6LPgwps7CgzpOKQw4eeacguH7gvzDyMqNIqmIU1pVq9ZsXFa\nA3wCOProUaLqReHr7RoPfBJFNpxsy0odRU+/dMY/6oZP+xTccMMN3HDDDc4Oo9ZVO+ErpbyBd4GB\nQDywWSm1TGv9t02x+4AzWut2SqkRwCvAHeXt92zuWZ748QnOm89fMDUJacLcQUW3Kvf7qB+ZeZml\nlp01YBYTe08E4Iu/vmDsd2PLPOatnW4lwMfotFm8azHrT6w3VpS4QurixhcXzocHhBMZHElYQBih\n/qE0CGxARFAEEYERFzxDZkb/GUy7ehoNAxvi71P+VR0uNYarKObjo5ezkNY0egx8Peh+M+H+HFHD\nvww4pLU+AqCU+hwYAtgm/CHATOv818BcpZTS5fQY5yfl0/+2/qWu8/byZm3A2sL3U3KmoLVm2p3T\n2N9yPwBjfxvLkM1DOHvqLPQ2yjX6oxHLZi8rrMUrpSj4h4It7xR1Er9gfgGLtvDXrX/R8PGGRkL/\nLRT1nCJ0SCjcbpSLOhPF1y9/XebJOW79Zyvz9kw6zu8IQPahbLb13UZg20B6/lnUVLOh1QbM58xl\n7reksrbvG9cXnxDjx7x7yG7S16VXep9Ase15DtbuX0vX77oS1s+o2R6dfpSE9xLs2mdp27d+vjVR\nD0UBcOqbU+yfsN+ufZa2faPbG11wnu1R2vb+rQP54phxO/7o0XbtTginc0TCjwJO2LyPB0q2cRSW\n0VqblFLpQEMg1baQUmo8MB6gRVALwnLK/nPZdK6oDTwUo4PyuXbPYe5uxlf5Um9XPQJzAgk5F1J4\nw1BwSjBkl/1BTBTt09v67yLvi/A/Y62NJ4A+o0k/nl50E9IJjEYqOyQdTiIpNqnY9pmBmcVvbDoF\n5FR+n2Vtv/aPtRBkXXbM/lhttzdnmCENtm/eDgXPUNtn/z5L2/7gXwc5GHvQWsD+fZa2fWnn2R6l\nbZ+hs8nID+LiJsdJSTlCDV095xRhYWGFd83aw2w2V2m7uqi2zkVubm6VboSs9mWZSqlhwI1a63HW\n9/cAfbTWD9uU2WMtE299f9haJrW0fQL0vLSn3vDTBrti8QnzwcvXuOXdlGXCkmvBO9gb70DjKgpz\nrhlzVuVrzQDrt64n5oaYYtt7+XvhU8/4XanNmvwz+eXs4UKlba+8FL4Nitrr89PysednU9b2vg18\nUV7GXzSmdBOW/IpvAbdlu33siliu6HtFqefZHjXxcypt+5r4OQ3/P8Wy1b5Mu2kF/1p1i137c3We\nfFmmo3jCZZkJQAub982ty0orE6+U8gHCqKC+pbwUfhFVbyD1CfGBEs+y8g7wxjvAzkvobJraS9te\neVcvzrK2921Yvc7a0rb3CavmjzuEC2It7Tzbw2E/pwq2d8TPKSXXj+Wx4OeVz01d/wLqVsIXdZ8j\nnpa5GWivlGqtlPIDRgDLSpRZBhS0eA4Dfiuv/V4IV/TJJ6A1DGmyidCQ884ORwi7VbuGb22Tfxj4\nEeOyzI+01n8ppV4AtmitlwEfAp8opQ4BpzF+KQjhNrSGRYuM+THNf0ErudnK04wYMQKTycTXX5d9\nkYarc8jz8LXWq7TWHbTWbbXW/7Yum25N9mitc7XWw7XW7bTWlxVc0SOEu9i4EQ4cgCZN4PqGW8Fb\nEr4rUEqVO9mORiXkTlshKqWgdn/PPeCzO7/YmLbCeZKSkgrnV6xYwf33319sWU09ddJdyYhXQlQg\nOxs+/9yYHz2aYgOgCOdq0qRJ4VS/fv0LloWFGZd2P/HEE7Rv357AwEBat27Nc889x/nzRf0wU6dO\npVevXixevJjWrVsTGhrKsGHDOHPmzAXHfPXVV2natCkNGjTg/vvvJy8v74Iyrkq+tUJU4NtvISMD\neveGiy+m2BCHwj2EhYWxePFi9u7dy9tvv83ChQt59dVXi5XZv38/y5cvZ/ny5axatYoNGzYwc+bM\nYmV+/vln4uLiWL16NUuWLOHzzz/nvffeq8VPUj3SpCNEBRYuNF7HFjyVwzrilSeo3Md0/HXnjr6G\nb8aMGYXz0dHRHD58mAULFvDcc88VK7do0aLCB6jde++9fPvtt8XWR0RE8M477+Dl5UWnTp249dZb\n+fXXX3n88ccdG3ANkYQvRDmOHYPffgN/fxhRcG2ZxSKdtm7ms88+45133uHIkSNkZWVhMpnw8yt+\nX0abNm2KPS2zWbNmFwxE0qVLF7xs/rpr1qwZ+/fb9xgQZ5K/S4Uox+LFRm3zttsgPNy60GLxmBq+\n1hVPGRmZlSpnz+RIsbGx3HPPPQwePJgVK1awfft2pk+fXqwNH4oPTg7GFUAlByevTBlXJjV8Icpg\nsRRdnTPW9iGrZrN02rqRdevW0bZtW6ZOnVq4LC4uznkBOZEkfCHKsGYNHDkCzZvDgAE2K6TT1q10\n6NCBo0eP8uWXX9KzZ09WrlzJN9984+ywnEK+tUKUoaCzdvToEk32HtRpWxcMGzaMSZMmMXHiRLp3\n787atWuLdeJ6EhnEvAIycHcRTzoXmZnGXbXZ2XDwILRrZ7OyVy+2jhtHzwcecFp8NUGelll9rv60\nTKnhC1GKr74ykv1VV5VI9iBNOsJtybdWiFJccO29Lem0FW5KvrVClHDwIKxdC8HBMHx4KQWkhi/c\nlHxrhSih4FLM4cMhpLTBXaTTVrgpSfhC2DCZyrj23pY8PE24KfnWCmFj5UpITISOHY0O21JJk45w\nU/KtFcLG/PnG6/jx5Tw4TDpthZuSb60QVsePw/ffg58fjBpVTkEPepaOqFsk4Qth9eGHxoO7hg6F\niIhyCkqTjnBT8q0VAqOz9sMPjfnx4ysoLE06wk3Jt1YIYNUqSEiA9u2hf/8KCksN36WMGTOmcNBy\nHx8fWrZsyYMPPljq8IRliY2NRSlFampqmce45ZZb7N7O1ci3Vggq2VlbQGr4Lue6664jKSmJuLg4\nFixYwPLly5k4caKzw3I58q0VHu/YMaOz1tfXOkh5RaTT1uX4+/vTpEkTmjdvzvXXX88dd9zBTz/9\nVLg+PT2d8ePH07hxY+rVq0f//v1xhYcz1jZJ+MLjvf++cS/V8OHQqFElNpAmHZd25MgRfvjhh8LR\nqbTW3HzzzSQkJBSOeHX11Vdz7bXXkpSU5ORoa5cMgCI8Wk4OfPCBMT9pUiU38rAmnVgVa1f5kEtD\n6LW16Om8BdvH6JjCZVt6biFrW1ap29uWq6wffviBkJAQzGYzubm5ALzxxhsArF69mh07dnDq1CkC\nAwMB+Ne//sXy5cv55JNPePrpp+0+nruShC882uefw+nT0LMn9OlTyY2kScflXH311cyfP5+cnBw+\n+OADDh8+zCOPPALA1q1byc7OplGJP99yc3M5fPiwM8J1Gkn4wmNpDe+8Y8xPmlSJztoCFkuJIbDq\ntopq3BUN+lHa9rZ/AThCUFAQ7awDF7z99ttcc801/Otf/2LmzJlYLBYiIyNZs2bNBduFhoZWav+h\noaGl/nI4e/YsXl5ebjMAjCR84bHWr4ft242brO64w44N5WmZLm/GjBncdNNNjB8/nksvvZTk5GS8\nvLxo06ZNlfbXsWNHPv30U3JycgqbhQC2bdtGq1at8Pf3d1ToNcpzGiKFKGHuXON1/HgICLBjQ+m0\ndXkxMTF07tyZF198keuuu45+/foxZMgQvv/+e44ePcqGDRuYMWPGBbX+PXv2sGPHjmKTxWLhrrvu\nwsfHh1GjRrF161YOHTrEwoULeeutt5g8ebKTPqX9pIYvPFJiInz9tdEyY/fQtB7WaeuunnzyScaO\nHcuUKVNYtWoV06ZN4/777yclJYXIyEj69evHqBIPTbrmmmsu2E9mZib169dnzZo1TJ06lcGDB5Oe\nnk67du144403uO+++2rrI1WbJHzhkd57z3icwtCh0KKFnRtLp61LWVQwgEEJI0eOZOTIkYXv58yZ\nw5w5c0otGxMTg9a63ON06NCBpUuXVjlOVyDVFOFxsrKMhA/wxBN2bqy1MUkNX7gh+dYKj/Phh3Dm\nDPTrB1dcYefGFotxOY/U8IUbkoQvPEp+Pljvx6FKfW0edkmmqFsk4QuP8uWXxkAnHTvCP/5RhR3I\neLbCjck3V3gMrWH2bGN+8uQq5u06foVORR2Xwvmq8zOq1jdXKdVAKfWzUuqg9TW8jHJmpdQO67Ss\nOscUoqp++gl27YImTeDuu6u4kzrcpOPr60tOTo6zwxAVyMnJKXwwnL2qW1WZCvyqtW4P/Gp9X5oc\nrXV36zS4mscUokpeftl4fewxqPKNkXW4ht+4cWMSEhLIzs6Wmr4L0lqTnZ1NQkICjRs3rtI+qnsd\n/hAgxjr/MRALTKnmPoVwuNhYYwoLgwkTqrGjOtyGX/BcmcTERPLz8yu9XW5uLgF23apcd9X0ufD1\n9SUyMrLSzwAqqboJP1JrXfBA6ZNAZBnlApRSWwAT8LLW+n+lFVJKjQfGA0RGRhIbG1vN8KovKyvL\nJeJwBe56LrSGxx7rDtTn9tuPsmPHsSrvyyc9nT5au+25qAlZWVmEhIQ4OwyXUBvnIj4+vuoba63L\nnYBfgD2lTEOAsyXKniljH1HW1zZAHNC2ouP27NlTu4LVq1c7OwSX4a7n4pdfjLulwsO1Tk+v5s6S\nk7WOiHDbc1ET5FwUcYVzAWzRZeTVCmv4WuvrylqnlEpWSjXVWicppZoCKWXsI8H6ekQpFQv0ADzr\nQdTCKbSG6dON+cmToYp/CRepw522ou6rbmPkMqBgFNDRwHclCyilwpVS/tb5CKAf8Hc1jytEpfz0\nk/EY5IgIePhhB+ywDnfairqvut/cl4GBSqmDwHXW9yileimlFljLXARsUUrtBFZjtOFLwhc1zrZ2\n//TT4JAxKupwp62o+6rVaau1TgMGlLJ8CzDOOr8e6Fqd4whRFd9+C5s2QePGMHGig3YqTTrCjUlV\nRdRJeXlFz8qZMQOCgx20Y2nSEW5MvrmiTnr7bThyBDp3Nka0chhp0hFuTL65os5JSYEXXzTmX38d\nfBw5zI806Qg3Jglf1DkzZkBGBtx4ozE5lDTpCDcm31xRp+zZA/PnG5Xw11+vgQNIDV+4MUn4os6w\nWODBB43XCROM9nuHkxq+cGPyzRV1xrx5sHYtREYWteE7nHTaCjcm31xRJ8THwxTrc1rnzoXwUkdm\ncABp0hFuTBK+cHtaGzdWZWbCkCEwdGgNHkyadIQbk2+ucHtffgnLlxsPRnv3XVCqBg8mTTrCjck3\nV7i1xMSih6LNng1RUTV8QGnSEW5MEr5wW2azMTZtaioMHAj3319LB5UavnBT8s0VbmvWLFi92rgq\n55NPaikPSw1fuDFJ+MIt/fEHzJxptNd/8omR9GuF1PCFG5NvrnA7p07ByJFGZXvqVKM5p9ZIp61w\nY/LNFW4lNxduvRUSEuCKK+D552s5AGnSEW5MEr5wG1rDvfcaQxa2aAFffw2+vrUchDTpCDcm31zh\nNmbOhM8+g5AQWLECmjZ1QhBSwxduTBK+cAsLF8ILLxiV6y++gG7dnBSItOELNybfXOHyPvkE7rvP\nmJ8zBwYNcmIw0qQj3Jh8c4VLW7IERo822u9ffLHorlqnkSYd4cYk4QuXVTLZP/ecsyNCavjCrck3\nV7gcreGVV+Cee4wK9b/+5SLJHqQNX7g1Rw7vLES1nT9vPOr4ww+N97Nnw+TJzo2pGGnSEW5MEr5w\nGampcMcd8NtvEBhoNOncfruzoypBmnSEG5OEL1zCr78aTThJSdCkCSxbBr17OzuqUkgNX7gxqaoI\npzp/3hiacOBAI9lfeSVs2uSiyR6khi/cmnxzhdOsWQM9exrt9EoZz8VZvdp4bILLkk5b4cakSUfU\nuuRkePppWLzYeN+mjTHfr59z46oUadIRbkwSvqg1Z87Am2/CW28ZA477+xuPN54yxeikdQvSpCPc\nmCR8UeNSU2HuXCPRp6cbywYNMh6T0K6dc2OzmzTpCDcmCV/UmC1bjET/+eeQl2csGzDAaKt3i+ab\n0kiTjnBjkvCFQ504YTzN8r//he3bjWVKGTX6KVPg6qudG1+1SZOOcGOS8EW1aA27d8P33xvPqF+7\ntmhd/frGUy4ffBDatnVejA4lNXzhxiThC7uYzbBnD6xbZ4w89fvvEB9ftD4gAP7xD7jzTrjpJuN9\nnSI1fOHGJOGLMmVnw969RoL/6y/47bduHDhgXGFjq0kTI7kPGgTXXw+hoc6Jt1ZIp61wY5LwPdj5\n85CSAomJEBcHR48WvR46BEeOGE02RRoA0Lq1MYB4v37Ga9euHpQDpUlHuLFqJXyl1HBgJnARcJnW\neksZ5W4E5gDewAKt9cvVOa4waA35+ZCTA+fOGZc8pqdDRkbRfMF09qyR3JOTjenkSeO6+PL4+ECH\nDtClizHBHsaN6+KcsWRdhTTpCDdW3Rr+HuB2YF5ZBZRS3sC7wEAgHtislFqmtf67vB2fOQNffllU\nw9S6/PnKlrN3mwMHoti92/7jmM1gMhW9ljVf1nqTCXJzjSknp2i+5PviNXD7eHtDo0bGYODR0cbU\nunXRa4cO4OdXVD42NtWzkz1Ik45wa9VK+FrrvQBKqfKKXQYc0lofsZb9HBgClJvwjxwxHpXrfO2d\nHUC5fMgn0CuPQK88wrwyCfPOIswry3j1ziw239j7NJG+aUT6nKaJTyoNvc/ipTSYgEPWqRy9zp2D\n4ODa+FiuKznZBcZZFKJqaqMNPwo4YfM+HuhTWkGl1HhgPECgXycu73oUpTSqcD0opW3KG/PKuo5i\n70usw1hfcl+Fv6rKOI7JZMLX17vYvorHYH1FF80rjbeXxstL4+1lwdvb+mpd5mNd5mVdVrDOdpmP\nlwU/XzP+fmbj1deMn4+p2HtfHzM+3pWp4vsBDa1Te3KBOOtkj+zsbIKCguzcqu7JadaMrKwsYmNj\nnR2KS5BzUcTVz0WFCV8p9QvQpJRVz2mtv3NkMFrr+cB8gF69eulft7R25O6rJDY2lpiYGGeH4RJi\nY2PpLecCkO+FLTkXRVz9XFSY8LXW11XzGAmA7QNvm1uXCSGEqEW10fu0GWivlGqtlPIDRgDLauG4\nQgghbFQr4SulblNKxQOXAyuVUj9alzdTSq0C0FqbgIeBH4G9wJda67+qF7YQQgh7VfcqnW+Bb0tZ\nnggMsnm/ClhVnWMJIYSoHrmgWAghPIQkfCGE8BCS8IUQwkNIwhdCCA8hCV8IITyEJHwhhPAQkvCF\nEMJDSMIXQggPIQlfCCE8hCR8IYTwEJLwhRDCQ0jCF0IID6F0dQZFrUFKqVPAMWfHAUQAqc4OwkXI\nuSgi56KInIsirnAuWmmtG5W2wmUTvqtQSm3RWvdydhyuQM5FETkXReRcFHH1cyFNOkII4SEk4Qsh\nhIeQhF+x+c4OwIXIuSgi56KInIsiLn0upA1fCCE8hNTwhRDCQ0jCF0IIDyEJ3w5KqSeVUlopFeHs\nWJxFKfWqUmqfUmqXUupbpVR9Z8dUm5RSNyql9iulDimlpjo7HmdRSrVQSq1WSv2tlPpLKfWos2Ny\nNqWUt1Jqu1JqhbNjKYsk/EpSSrUArgeOOzsWJ/sZ6KK17gYcAJ5xcjy1RinlDbwL3AR0Bu5USnV2\nblROYwKe1Fp3BvoCD3nwuSjwKLDX2UGURxJ+5b0JPA14dC+31vonrbXJ+nYj0NyZ8dSyy4BDWusj\nWuvzwOfAECfH5BRa6ySt9TbrfCZGootyblTOo5RqDtwMLHB2LOWRhF8JSqkhQILWeqezY3Ex9wLf\nOzuIWhQFnLB5H48HJrHjfAAAAWlJREFUJ7kCSqlooAfwp3Mjcaq3MCqEFmcHUh4fZwfgKpRSvwBN\nSln1HPAsRnOORyjvXGitv7OWeQ7jz/pPazM24VqUUiHAN8BjWusMZ8fjDEqpW4AUrfVWpVSMs+Mp\njyR8K631daUtV0p1BVoDO5VSYDRhbFNKXaa1PlmLIdaass5FAaXUGOAWYID2rBs5EoAWNu+bW5d5\nJKWUL0ay/1RrvdTZ8ThRP2CwUmoQEACEKqWWaK3vdnJcF5Abr+yklIoDemmtnf1EPKdQSt0IvAH0\n11qfcnY8tUkp5YPRUT0AI9FvBkZqrf9yamBOoIzaz8fAaa31Y86Ox1VYa/hPaa1vcXYspZE2fGGv\nuUA94Gel1A6l1H+cHVBtsXZWPwz8iNFJ+aUnJnurfsA9wLXW78EOaw1XuDCp4QshhIeQGr4QQngI\nSfhCCOEhJOELIYSHkIQvhBAeQhK+EEJ4CEn4QgjhISThCyGEh/h/tkk7It+SZ1MAAAAASUVORK5C\nYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3lhNBrSyxthW",
        "colab_type": "text"
      },
      "source": [
        "ReLU activation functions are good because they dont saturate for positive values and is fast to compute. However they can cause some neurons to die (*dying ReLUs*), so they only output 0, if weights are updated such that the weighted sum of the neurons inputs are negative. This can be helped by *leaky* ReLU, where a slope is defined by a hyperparameter to ensure they do not die<sup>1,2</sup>. \n",
        "\n",
        "**NOTES**\n",
        "- code from https://github.com/ageron/handson-ml/blob/master/11_deep_learning.ipynb\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. Xu, B., Wang, N., Chen, T., & Li, M. (2015). Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7amJRu2ExuHW",
        "colab_type": "code",
        "outputId": "097078fc-72b1-4eb5-ec29-f93493a5d445",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "source": [
        "# for leaky_relu\n",
        "def leaky_relu(z, alpha=0.01):\n",
        "    return np.maximum(alpha*z, z)\n",
        "\n",
        "z = np.linspace(-5, 5, 200)\n",
        "\n",
        "plt.plot(z, leaky_relu(z, 0.05), \"b-\", linewidth=2)\n",
        "plt.plot([-5, 5], [0, 0], 'k-')\n",
        "plt.plot([0, 0], [-0.5, 4.2], 'k-')\n",
        "plt.grid(True)\n",
        "props = dict(facecolor='black', shrink=0.1)\n",
        "plt.annotate('Leak', xytext=(-3.5, 0.5), xy=(-5, -0.2), arrowprops=props, fontsize=14, ha=\"center\")\n",
        "plt.title(\"Leaky ReLU activation function\", fontsize=14)\n",
        "plt.axis([-5, 5, -0.5, 4.2])\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEJCAYAAAC9uG0XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de3hU1b3/8feXgBAgCJWCiPzEW1XE\nIkhtkSNQvFRRaq3WIyqKVoNW6uUoavGCVakWxSPFG1AURS56qHexPShE5WhRRCxFgSpQEVFUiBAw\nCUnW7481MRdymYTMrD0zn9fzzMOe2Tt7f/dm8smaNWvvbc45REQkupqFLkBEROqmoBYRiTgFtYhI\nxCmoRUQiTkEtIhJxCmoRkYhTUKcAM8szs/tD15EOzGyQmTkz65iEba0zs2uTsJ1DzewtMys0s3WJ\n3l4c9TgzOzN0HelEQb2bzGy6mb0Yuo6GioW/iz2KzexjM7vTzFo2cD0jzKygnu3s8kemvp9rCrUE\n5ZtAF+DrJtzOrWb2zxpm/Qh4sKm2U4c7gB3AobFtJkUd7/0uwAvJqiMTNA9dgAT1KDAG2AP/C/5o\n7PXfBasowZxzxcDnSdrWl8nYDnAQ8Jxzbl2Stlcn51xSjm8mUYs6wcxsTzObYmabzGybmb1mZn0r\nzd/LzGab2adm9q2ZrTCzC+tZ53Fmlm9ml5rZADPbaWZ7V1tmnJn9o57ydjjnPnfOfeKc+wswHzix\n2nq6mtkcM9sSe7xkZgc38DA0ipndZWarYsdlnZmNN7NW1ZYZYmaLY8t8bWYvmFkrM8sD9gPuLv/k\nEFv+u64PM2sX+7mh1dZ5YuyYdqqvDjMbAYwFDq/0CWVEbF6VFr2Z/T8zeyb2PthmZk+b2b6V5t9q\nZv80s7Njn3C2mdmzdXXTxParF3BLbNu3mln32HTf6suWd0lUWuYMM5tvZjvM7AMzO6HazxxqZs+b\n2TdmVhDrYjnCzG4FLgBOqbTfg6pvJ/b8CDN7JXb8Nsda4ntWmj/dzF40syvNbEPsffaombWubb8z\njYI6gczMgJeArsCpQG/gdWCBmXWJLdYKWBqbfzgwEZhsZsfVss4zgWeAXOfcw86514GPgfMrLdMs\n9nxaA2rtBfQHdlZ6rTWwECgEBgL9gI3AK0n6JdoOXAQcBvwGOBu4sVJ9JwHP4//AHAX8FHgN/77+\nJfApcBv+o3gXqnHObcV/RD+32qxzgfnOuU1x1PEkMAFYVWk7T1bfVuz/5Dmgc6zOnwL7AM/G3ifl\nugP/CZyO/6PZGxhXy/Ehtr1VsRq6APfUsWxNxgF/wof9O8AcM2sbq3kfYBHggBOAPsADQFZsO08B\nr1Ta7zdr2O82wN+AAuDo2H4dAzxSbdFjgZ7A8VTs/5UN3Jf05ZzTYzcewHTgxVrmDca/QbOrvb4M\nuK6Odc4B/lzpeR5wP5ALfAOcWG35a4EPKz0/GSgC9qpjG3lAcay+IvwvYylwRqVlLgL+BVil17Lw\n/btnxZ6PAArq2c79Nbxe58/Vsq5LgY8qPf8/YE4dy68Drq322qDYvnaMPf85vn83J/Y8G9gKnNOA\nOm4F/lnX9vFBVwp0rzT/AKAMOL7SegqBPSstc2PlbdVSzz+BWys97x7bx77VlnPAmdWWGVlpftfY\na/8Rez4O+DewR0Pe+9W2c0nsPZtTw//BQZXWsx7IqrTMVOCVxvxOpuNDLerEOgpoDXwZ+9hYYP4L\ntJ7AgQBmlmVmN5rZP2If3QvwrcH/V21dv8C3Zk5yzv1vtXmPAQeY2TGx5xcBzzrn6vvC7EngSHxL\n+SlgqvNdIJXr3x/YVqn2b4AO5fUnkpmdaWaLzOzz2Lb/m6rHpTfw6m5u5mV8UJ8ee/5zwIBnG1BH\nPA4DPnOV+pGdc2uAz4AelZb7t3Pum0rPPwM6NXBbDVG5e+yz2L/l2+sNLHK+X7+xDgP+4ZzbVum1\nN/F/oCrv9wfOudJqtSRyv1OKvkxMrGbAF/iPddVtjf17LXAN/mPecnwL9w/s+iZ9HzgC+LWZ/d3F\nmh3gv7Qys+eBi8xsFT5shlK/b5xzHwGY2XnACjMb4ZybXqn+ZfiP+tVtjmP94Pdzzxpeb48P/RqZ\n2U/wnyx+D1wN5OP3q6Ef7evknNtpZk/huzsej/37jHNuRxLrqHwJy501zGtog6os9u93XSpm1qKW\nZb/bnnPOxXphktWAa+r9TlsK6sRaiu+TLIu1nmryH8ALzrkZ8F2/9g/wgVDZWuC3+K6EKWaWWzms\n8R8V5wJr8KMaXmlIobHA+gNwp5k9FQuqpcAw4CvnXPV64rUKGGJmVq3ePrF5tekPbHDO3V7+gpnt\nV22Z94Dj8Ptek2J8V019ngBeN7MewEn47wsaUkc82/kQ2MfMupe3qs3sAHw/9Qdx1NgQ5aNNKvfL\nH9mI9bwHnGdme9TSqo53vy8ys5xKrepj8CH8YSNqykj6i9U02pnZkdUe3fFh+X/Ac2Z2spntb2b9\nzOz3Zlbeyl4NHGdm/2Fmh+L7ovevaSOxsP8pPkwmV/sSaj6+73gsMN05V1bDKuozC9+SGRV7PhP/\nieA5MxsYq3+AmU2wqiM/mtWw/z1j8x7C98VOMrNeZnaImV2N/wNwdx21rAa6mtm5ZnaAmV0W+5nK\nxgG/MrM7zKyHmR1uZldX+qJzHXCs+ZErtY6ccM69ie+LnQV8RdXulHjqWAfsZ2Z9zI8mqWks+iv4\nboaZZtbX/IiMmfg/hgvqOA4N5pz7Fvg7cH3smBxD4z4BPAi0BZ4ysx+Z2UFmNszMykN/HdAz9n/a\nsZZW+0x819Lj5kd/DAAmA0+Xf5qT+imom8ax+NZH5cc9sRbkEPwv4lR8C/Ip4BAq+gPvAN7G95W+\njh9hMLO2DTnnPsZ/GXMylcI6tq1HgRZUjIdukFir6X7gulgLaAcwAN9K/x9gJb4/vAOwpdKPZtew\n/3mxda6JreNg4H9j+3o28Cvn3Mt11PICPsjvwwfcCcAt1ZaZh+9bPjm2zdfwf8jK/0jdAnTDj4qp\nb0zzTPzIhzmV+0rjqQP4CzAPH/BfsmuQl///nBabvzD2+Bz4RbVPGk3loti/7+CD8aaGrsA5twH/\nf7cHvt738J/qSmKLTMW3ipfg96t/DevYAfwMaIf/v38OeKtSfRIHS8x7REIws4fw36SfUO/CIpIy\n1EedBsyfPNADP3b6rMDliEgTU1Cnh+fwJxNMc869FLoYEWla6voQEYk4fZkoIhJxCen66Nixo+ve\nvXsiVh237du306ZNm6A1RIWOhbdq1SpKS0vp0aNH/QtnAL0vKtR0LFavhm3boF07ODgJlyF79913\nv3LOfb+meQkJ6u7du7NkyZJErDpueXl5DBo0KGgNUaFj4Q0aNIj8/Pzg782o0PuiQvVjceedMGYM\ndOoE//gHdO6c+BrM7N+1zVPXh4hIJYsXw803++nHHktOSNdHQS0iEvPNNzBsGJSWwn/9F5x0UuiK\nPAW1iAjgHPzmN7B2LfTuDX/4Q+iKKiioRUSAGTNg1ixo3Rpmz4aWDbp7aGLFHdSx6ya/Zyl4I1cR\nkbps2JDN5Zf76UmT4JBDwtZTXUNa1FeiyxKKSJopLobbbz+MggL4z/+EC+u8Y2kYcQW1+RtwngL8\nObHliIgk1003wapV7dhvP3j4Yahy8eCIiLdFfR9wHRWXjxQRSXnz58Pdd0OzZo5Zs6B9+9AV1aze\nE17M7FRgk3PuXYvdDr6W5XLxN1+lc+fO5OXlNVWNjVJQUBC8hqjQsfDy8/MpLS3VsYjJ9PdFfn4L\nfv3rvkBLhg1bTXHxRqJ6OOI5M7E/8HMzGwK0wt/N5Ann3HmVF3LOTQGmAPTt29eFPuNJZ11V0LHw\n2rdvT35+vo5FTCa/L5yDU0+FzZthwAC48MKNkT4W9XZ9OOd+55zb1znXHX9njgXVQ1pEJJX86U8w\nbx506ABPPAFZ8dxZMyCNoxaRjLJsGVx3nZ+eNg26dQtbTzwadFEm51wesXvhiYikmu3b/SnixcUw\nciScfnroiuKjFrWIZIyrr4aVK6FHD7j33tDVxE9BLSIZYe5cmDrVnxo+Z44/VTxVKKhFJO198glc\ncomfvuceOOKIsPU0lIJaRNJaSQmcey7k58PQoXx3TY9UoqAWkbQ2bhwsWgRdusAjj0TzFPH6KKhF\nJG298QbcdpsP5yeegI4dQ1fUOApqEUlLW7b4Lo+yMrj+ehg8OHRFjaegFpG04xzk5sL69XD00b5V\nncoU1CKSdqZN88PxcnL83VpatAhd0e5RUItIWvnwQ7jiCj/90ENwwAFh62kKCmoRSRuFhf4U8W+/\nheHDfR91OlBQi0jauOEGeP99OOggeOCB0NU0HQW1iKSFl16CiROheXN/N/GcnNAVNR0FtYikvI0b\nYcQIPz1uHPzoR0HLaXIKahFJaWVlcP758NVXcPzxcO21oStqegpqEUlpEybAK6/4sw4ffxyapWGq\npeEuiUimeOcdGDPGT0+f7q/nkY4U1CKSkrZt80PxSkr8uOlTTgldUeIoqEUkJY0aBR9/DL16wR//\nGLqaxFJQi0jKmTnT90dnZ/tTxFu1Cl1RYimoRSSlrFkDl13mpydOhMMOC1tPMiioRSRl7Nzp+6W3\nbYMzzoCLLw5dUXIoqEUkZYwdC2+/Dd26+RvVpuLdWhpDQS0iKWHBArjrLj9OeuZM6NAhdEXJo6AW\nkcj76is47zx/Q4Cbb4Zjjw1dUXIpqEUk0pyDiy7y1/Po3x9uuil0RcmnoBaRSHvwQXjhBdhzT9/l\n0bx56IqST0EtIpG1fDlcc42fnjoV9tsvbD2hKKhFJJJ27ICzz4aiIj8M71e/Cl1ROApqEYmka66B\nDz6AQw+F++4LXU1YCmoRiZxnnoGHH4Y99vCniLdpE7qisBTUIhIp69fDr3/tp8ePhyOPDFtPFCio\nRSQySkv93cO3bIEhQ/zlS0VBLSIRcued8Npr0LkzPPpo5pwiXh8FtYhEwltvwa23+ukZM6BTp6Dl\nRIqCWkSCy8/3V8UrLYXRo+GEE0JXFC0KahEJyjm49FL497+hb1+4447QFUWPglpEgpo+HZ580g/B\nmzXLD8mTquoNajNrZWZvm9n7ZrbCzH6fjMJEJP2tWgW//a2ffvBBOPjgsPVEVTyXNykCBjvnCsys\nBbDIzF52zv09wbWJSBorKvL90tu3wznn+GF5UrN6g9o554CC2NMWsYdLZFEikv7GjIH33oP994eH\nHtJQvLrEdcFAM8sC3gUOAh5wzi2uYZlcIBegc+fO5OXlNWGZDVdQUBC8hqjQsfDy8/MpLS3VsYgJ\n+b54++3vce+9P6RZM8e1177H0qVbg9RRLvK/I865uB9Ae2Ah0LOu5Y466igX2sKFC0OXEBk6Ft7A\ngQNdr169QpcRGaHeF59/7lynTs6Bc3/4Q5ASdhGF3xFgiaslUxs06sM5lx8L6pOa/C+GiKS9sjK4\n4ALYtAl++lO47rrQFaWGeEZ9fN/M2sems4ETgJWJLkxE0s9998Hf/gZ77eXPPszKCl1Raoinj7oL\n8Fisn7oZ8JRz7sXEliUi6WbpUrjhBj89bRp07Rq2nlQSz6iPfwC9k1CLiKSpggI/FG/nTrj8cjjt\ntNAVpRadmSgiCXfFFbB6NfTsCXffHbqa1KOgFpGEevJJf8nSVq1gzhzIzg5dUepRUItIwqxbB7m5\nfvree+Hww4OWk7IU1CKSECUl/tTwrVvhF7/wV8iTxlFQi0hC/P73/mYAXbvCn/+sU8R3h4JaRJrc\na6/BuHE+nJ94wo+blsZTUItIk9q8Gc47z98Q4MYbYdCg0BWlPgW1iDQZ5+Dii+HTT6FfPxg7NnRF\n6UFBLSJNZvJkeOYZaNfO362leVzX55T6KKhFpEmsWAFXX+2nJ0+G7t2DlpNWFNQistsKC/0p4oWF\ncOGFcPbZoStKLwpqEdlto0fD8uXwgx/An/4Uupr0o6AWkd3y/PNw//3QogXMng1t24auKP0oqEWk\n0TZsgIsu8tN33gl9+oStJ10pqEWkUUpL4fzz4euv4Wc/q/giUZqeglpEGuXuu2HBAujUCR57DJop\nTRJGh1ZEGmzxYrjpJj/92GPQuXPYetKdglpEGmTrVj8Ur7TUd3ecpFtdJ5yCWkTi5hxcdhmsXQu9\ne/svECXxFNQiErcZM/yp4a1b+6F4LVuGrigzKKhFJC4ffeRvTAswaRIcckjYejKJglpE6lVc7Pul\nCwrgrLP8aeKSPApqEanXzTfDkiWw337+gku6W0tyKahFpE7z58P48ZCV5fun27cPXVHmUVCLSK2+\n/NKffQj+JgDHHBO2nkyloBaRGjnn+6I//xwGDIAxY0JXlLkU1CJSo0mT4KWXoEMHf4ParKzQFWUu\nBbWI7GLZMn+NaYBp06Bbt7D1ZDoFtYhUsX27H4pXXAwjR8Lpp4euSBTUIlLF1VfDypXQowfce2/o\nagQU1CJSydy5MHWqPzV8zhx/qriEp6AWEQA++QQuucRP33MPHHFE2HqkgoJaRCgpgXPPhfx8GDq0\n4poeEg0KahFh3DhYtAi6dIFHHtEp4lGjoBbJcIsWwW23+XB+4gno2DF0RVKdglokg23ZAuecA2Vl\ncP31MHhw6IqkJgpqkQzlHOTmwvr1cPTRvlUt0VRvUJtZNzNbaGYfmNkKM7syGYWJSGLNm9eFuXMh\nJ8dfFa9Fi9AVSW2ax7FMCXCNc26pmeUA75rZfOfcBwmuTUQS5MMP4f77DwLgoYfgwAMDFyR1qrdF\n7Zzb6JxbGpveBnwIdE10YSKSGIWF/hTxwsIshg/3w/Ik2uJpUX/HzLoDvYHFNczLBXIBOnfuTF5e\n3u5XtxsKCgqC1xAVOhZefn4+paWlGX8s7r//IN5/f1+6dNnO2WcvJS+vNHRJwUX9dyTuoDaztsBf\ngKucc1urz3fOTQGmAPTt29cNGjSoqWpslLy8PELXEBU6Fl779u3Jz8/P6GMxbx785S/QvDnccstK\nhgw5NnRJkRD135G4Rn2YWQt8SM90zj2d2JJEJBE2boQRI/z0uHFw6KHbgtYj8Ytn1IcB04APnXO6\nlpZICior87fU+vJLOP54uPba0BVJQ8TTou4PDAcGm9my2GNIgusSkSY0YQK88oo/6/Dxx6GZzqBI\nKfX2UTvnFgE6818kRb3zTsX9DqdP99fzkNSiv6siaWzbNj8Ur6QErrgCTjkldEXSGApqkTQ2ahR8\n/DH06gV//GPoaqSxFNQiaWrWLN8fnZ0Ns2dDq1ahK5LGUlCLpKE1a+DSS/30xIlw2GFh65Hdo6AW\nSTM7d/p+6W3b4Iwz4OKLQ1cku0tBLZJmxo6Ft9+Gbt38jWp1t5bUp6AWSSMLFsBdd/lx0jNnQocO\noSuSpqCgFkkTX30Fw4f7GwLcfDMcq8t4pA0FtUgacA4uugg++wz694ebbgpdkTQlBbVIGnjwQXjh\nBdhzT9/l0bxBFzCWqFNQi6S45cvhmmv89NSpsN9+YeuRpqegFklhO3b4oXhFRX4Y3q9+FboiSQQF\ntUgKu+YaWLECDj0U7rsvdDWSKApqkRT1zDPw8MOwxx7+FPE2bUJXJImioBZJQZ9+WnHG4fjxcOSR\nYeuRxFJQi6SY0lI47zzYvBmGDPGXL5X0pqAWSTF33gmvvQadO8Ojj+oU8UygoBZJIW+9Bbfe6qcf\nfxw6dQpajiSJglokRXzzDZxzju/6GD0aTjwxdEWSLApqkRTgHIwcCevWQd++cMcdoSuSZFJQi6SA\n6dPhySf9ELxZs/yQPMkcCmqRiFu9Gn77Wz/9wANw8MFh65HkU1CLRFhRkT9FfPt23z99/vmhK5IQ\nFNQiEXbjjbB0Key/Pzz0kIbiZSoFtUhE/fWvMGECZGX5ful27UJXJKEoqEUi6Isv4IIL/PRtt8FP\nfhK2HglLQS0SMWVlMGIEbNoEP/0pXH996IokNAW1SMTcd5/v9thrL5gxw3d9SGZTUItEyNKlcMMN\nfnraNOjaNWw9Eg0KapGIKCjwQ/F27oTLL4fTTgtdkUSFglokIq680p/c0rMn3H136GokShTUIhHw\n5JPwyCPQqhXMmQPZ2aErkihRUIsEtm4d5Ob66XvvhcMPD1qORJCCWiSgkhJ/avjWrfCLX8Cll4au\nSKJIQS0S0G23+ZsBdO0Kf/6zThGXmimoRQJ57TV/XWkzeOIJP25apCYKapEANm/2N6h1DsaMgUGD\nQlckUVZvUJvZI2a2ycz+mYyCRNKdc3DxxfDpp9CvH4wdG7oiibp4WtTTgZMSXIdIxpgyBZ55xl8N\nb9YsaNEidEUSdfUGtXPudWBzEmoRSXsrVsBVV/npyZOhe/eg5UiKaN5UKzKzXCAXoHPnzuTl5TXV\nqhuloKAgeA1RoWPh5efnU1paGuxYFBc347LL+lBY2JaTTtrI3nuvIuR/i94XFaJ+LJosqJ1zU4Ap\nAH379nWDAn87kpeXR+gaokLHwmvfvj35+fnBjsVvfwtr1vh7Hv7P/3ShbdsuQeoop/dFhagfC436\nEEmCF16A++/3/dFz5kDbtqErklSioBZJsA0b4MIL/fSdd0KfPmHrkdQTz/C82cBbwCFm9qmZ/Trx\nZYmkh9JSf+fwr7+GE0+Eq68OXZGkonr7qJ1zw5JRiEg6uvtuWLAAOnWCxx6DZvoMK42gt41Igixe\nDDff7Kcfewz23jtsPZK6FNQiCbB1q79bS0mJ7+44SaeMyW5QUIskwG9+A2vXQu/e/gtEkd2hoBZp\nYjNmwMyZ0Lo1zJ4NLVuGrkhSnYJapAl99JFvTQNMmgSHHBK2HkkPCmqRJlJc7PulCwrgrLMqxk6L\n7C4FtUgTuflmWLIE9tvPX3BJd2uRpqKg3k2DBg1i1KhRocuQwObPh/HjISvLX7q0ffvQFUk6Sfug\nHjFiBKeeemroMiSNffmlP/sQ/E0AjjkmbD2SftI+qEUSyTnfF/355zBggL+tlkhTy+ig/uabb8jN\nzaVTp07k5OQwcOBAlixZ8t38r7/+mmHDhrHvvvuSnZ3N4YcfzqOPPlrnOl999VXat2/Pww8/nOjy\nJQImTYKXXoIOHfwNarOyQlck6Shjg9o5xymnnMKGDRt48cUXee+99xgwYACDBw9m48aNABQWFtKn\nTx9efPFFVqxYwZVXXsnIkSN59dVXa1zn3LlzOf3005kyZQqXXnppMndHAnj/fRg92k9PmwbduoWt\nR9JXk904INUsXLiQZcuW8eWXX5KdnQ3A7bffzgsvvMCMGTO47rrr6Nq1K6PLfxOB3NxcFixYwOzZ\nsznuuOOqrG/KlCmMHj2auXPncuKJJyZ1XyT5tm+Hs8/2Q/JGjoTTTw9dkaSzjA3qd999lx07dvD9\n73+/yuuFhYV8/PHHAJSWlnLXXXfx5JNPsmHDBoqKiiguLt7lThDPPvsskydP5vXXX6dfv37J2gUJ\n6OqrYeVK6NED7r03dDWS7jI2qMvKyujcuTNvvPHGLvPatWsHwD333MOECROYOHEiRxxxBG3btmXM\nmDFs2rSpyvK9evVi+fLlTJs2jZ/85CeYBtCmtblzYepUf2r47Nn+VHGRRMrYoO7Tpw9ffPEFzZo1\n44ADDqhxmUWLFjF06FCGDx8O+H7t1atX077aINn999+fSZMmMWjQIHJzc5kyZYrCOk198glccomf\nvuce+OEPw9YjmSEjvkzcunUry5Ytq/I46KCD6N+/P6eddhovv/wya9eu5a233mLs2LHftbJ/8IMf\n8Oqrr7Jo0SJWrlzJqFGjWLt2bY3bOOCAA1i4cCF//etfGTlyJM65ZO6iJEFJCZx7LuTnw9ChcPnl\noSuSTJERQf3GG2/Qu3fvKo/Ro0czb948Bg8ezCWXXMIhhxzCWWedxapVq9hnn30AuOmmmzj66KM5\n+eSTGTBgAG3atOHcc8+tdTsHHnggeXl5vPzyywrrNDRuHCxaBF26wCOP6BRxSZ607/qYPn0606dP\nr3X+xIkTmThxYo3zOnTowNNPP13n+vPy8qo8P/DAA1m/fn1Dy5SIW7QIbrvNh/OMGdCxY+iKJJNk\nRItaZHds2eK7PMrK4PrrodrITJGEU1CL1ME5yM31XyIefbRvVYskm4JapA7TpvnheDk5/qp4LVqE\nrkgykYJapBYrV8KVV/rphx6CAw8MW49krpQN6rVr1zJ06NBah8uJ7I7CQn+K+I4dMHy476MWCSUl\ng/qdd96hT58+vPzyywwcOJAtW7aELknSzA03+IsuHXggPPBA6Gok06VcUD/33HMMGjSI/Px8SktL\n+eKLLzjhhBMoKioKXZqkiXnzYOJEaN7cnyKekxO6Isl0KRXUEydOZNiwYezYseO714qLi1m+fDk3\n3nhjwMokXWzcCCNG+Olx4+BHPwpajgiQIie8lJWVcdVVVzFt2jS+/fbbKvOysrLIyclhRPlvl0gj\nlZXBBRf4W2sdfzxce23oikS8yAd1YWEhZ555JgsXLqzSkgZo2bIl3bp1Iy8vj65duwaqUNLFhAn+\nJrUdO8Ljj0OzlPq8Keks0kH99ddfc/zxx7Ny5UoKCwurzMvOzqZv37689NJL5KgTUXbTkiUV9zuc\nPt1fz0MkKiLbZvj444/p1asXK1as2CWkW7duzRlnnMGrr76qkJbdtm0bDBvmr453xRVwyimhKxKp\nKpJB/fe//52jjjqKzz77jJ07d1aZl52dzXXXXcfjjz9OC50mJk1g1Cj46CPo1Qv++MfQ1YjsKnJd\nH08//TTDhw/fpT8afEhPnjz5uwv5i+yuWbN8f3R2th+K16pV6IpEdhWpFvWECRM477zzagzptm3b\nMm/ePIW0NJk1a6D8ZvETJ8Jhh4WtR6Q2SQ/qiRMnMnz48CoX1S8tLeWyyy7jlltu2WX4XfPmzenU\nqROLFy/e5aayIo21cyecc47vnz7jDLj44tAVidQuqV0fpaWl3H777RQUFNClSxfGjx/Pjh07+OUv\nf8kbb7xR4/C77t27k5eXx957753MUiXNjR0LixdDt27+RrW6W4tEWVKDet68eRQVFVFUVMQDDzzA\nXnvtxcyZM/nXv/5V48iOH//4xzz//PO0bds2mWVKmluwAO66y4+TnjkTOnQIXZFI3ZIa1OPHj6eg\noACAHTt2cMstt+Cc22VkR3Fr0LEAAAczSURBVOvWrTnrrLOYOnUqzZtH7vtOSWElJcbw4f6GALfc\nAsceG7oikfrF1UdtZieZ2Soz+8jMbmjMhtauXcuSJUuqvFZcXFzj8LsxY8bwyCOPKKSlSTkH69e3\n5rPPoH9/uOmm0BWJxKfeJDSzLOAB4ATgU+AdM3veOfdBQzY0adIkSktL61wmOzubadOmMWzYsIas\nWqRGRUX+foebN8OmTbBsGWzd2oI99/RdHmoHSKqwyqMvalzArB9wq3PuZ7HnvwNwzt1Z28/k5OS4\no4466rvnZWVlvPnmm/UGdc+ePdlrr73ir74O+fn5tG/fvknWlepS/ViUlFQ8du6s+d+aXisrq76m\nZQAceeSR7Lln0ncjclL9fdGUonAsXnvttXedc31rmhdPm6IrsL7S80+BH1dfyMxygVyAFi1akJ+f\n/928zZs3E8cfBNasWYOZ0awJroZTWlpapYZMFoVj4RyUljajpMQoLfWPytNVn1ddbnc0b+7Iyioj\nK8tRXOxo0aIU5/LRWyMa74uoiPqxaLIPf865KcAUgL59+7rK/dFHHnkk69evr+1Hy38e5xyHHXYY\nc+bMwXZzvFReXp7GXcc01bFwzo873rzZP8q7FeKZ3r698dvNyYHvfc8/OnSIf7pNm6rD7spvOLFs\n2bLdPhbpQL8jFaJwLOrKvHiCegPQrdLzfWOvxWX58uWsXr06rmV37tzJU089xQUXXMCQIUPi3YQ0\nUHGxD9CGBG35dD29V7Vq3rzhQfu970H79rrzt0g8Qf0OcLCZ7Y8P6LOBc+LdwH333UdxcXGN85o1\na0bbtm359ttv6dGjBz//+c858cQT6devX7yrz1jOQUFBfOG6Zk0vysoqnsdGSDZK27YNC9ry523b\n6qQSkcaqN6idcyVmNgr4G5AFPOKcWxHPyrdt28bs2bOrfInYrl07vv32W7p3787QoUP52c9+Rv/+\n/WnTpk1j9yGl7dxZd+u2thDessV/YRafqmd0ZGU1vnW7xx5NfghEpB5x9VE75+YB8xq68jlz5lBU\nVETLli3p1KkTJ598MieffDIDBw6kQxqdDuac74NtSNCWT2/b1vjttmkTX9B+8skyBg8+8rvXc3LU\nuhVJJQkdSdqvXz9mzJjB4MGDU+JaHSUlu7Zu4+2/jb91W1WzZo1r3XboEH/rNi8vn969G1efiISX\n0KDu2bMnPXv2TOQmdlHeut20qSXvv9+wL8q2bm38dlu3blzfbU6O7s0nInWL7LlZJSWQn9/wYWCb\nN/t+X2j4F5LNmvnwbEjQlv/bsmWTHwIRESDBQe0c7NjRuGFg33zT+O1mZ0ObNkV06dKyQaHbrp1a\ntyISPQkJ6hUr/F2cN2/2Y3Ybw6zxrdtWrSAv763gA9hFRJpCQoK6sBA+/9xPt2rVsKAtn95zT7Vu\nRUQgQUHdowfMn+8DNzs7EVsQEckcCQnq7GzYZ59ErFlEJPOoc0FEJOIU1CIiEaegFhGJOAW1iEjE\nKahFRCJOQS0iEnEKahGRiFNQi4hEnIJaRCTizDnX9Cs1+xL4d5OvuGE6Al8FriEqdCwq6FhU0LGo\nEIVjsZ9z7vs1zUhIUEeBmS1xzvUNXUcU6FhU0LGooGNRIerHQl0fIiIRp6AWEYm4dA7qKaELiBAd\niwo6FhV0LCpE+likbR+1iEi6SOcWtYhIWlBQi4hEXEYEtZldY2bOzDqGriUUM7vbzFaa2T/M7Bkz\nax+6pmQys5PMbJWZfWRmN4SuJxQz62ZmC83sAzNbYWZXhq4pNDPLMrP3zOzF0LXUJu2D2sy6AScC\nn4SuJbD5QE/n3A+B1cDvAteTNGaWBTwAnAz0AIaZWY+wVQVTAlzjnOsB/AS4PIOPRbkrgQ9DF1GX\ntA9q4L+B64CM/tbUOfe/zrmS2NO/A/uGrCfJjgY+cs6tcc4VA3OA0wLXFIRzbqNzbmlsehs+oLqG\nrSocM9sXOAX4c+ha6pLWQW1mpwEbnHPvh64lYi4CXg5dRBJ1BdZXev4pGRxO5cysO9AbWBy2kqDu\nwzfkykIXUpeE3IU8mczsFWDvGmbdCIzBd3tkhLqOhXPuudgyN+I//s5MZm0SLWbWFvgLcJVzbmvo\nekIws1OBTc65d81sUOh66pLyQe2cO76m183sCGB/4H0zA/9Rf6mZHe2c+zyJJSZNbceinJmNAE4F\njnOZNYB+A9Ct0vN9Y69lJDNrgQ/pmc65p0PXE1B/4OdmNgRoBbQzsyecc+cFrmsXGXPCi5mtA/o6\n50JfISsIMzsJuBcY6Jz7MnQ9yWRmzfFfoB6HD+h3gHOccyuCFhaA+VbLY8Bm59xVoeuJiliL+lrn\n3Kmha6lJWvdRSxX3AznAfDNbZmYPhy4oWWJfoo4C/ob/8uypTAzpmP7AcGBw7H2wLNailAjLmBa1\niEiqUotaRCTiFNQiIhGnoBYRiTgFtYhIxCmoRUQiTkEtIhJxCmoRkYj7/xa1GvPKeAb9AAAAAElF\nTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b--mK4gOxxSW",
        "colab_type": "text"
      },
      "source": [
        "A variation on ReLU can also be used such as the *exponential linear unit (ELU)* which can take on negative values to alliviate vanishing gradients, has a nonzero gradient to avoid dying ReLUs, and can speed up Gradient Descent if the function is smooth<sup>2</sup>. It is still slower than ReLU to compute at testing time, but during testing has a faster convergence rate<sup>1</sup>.\n",
        "\n",
        "**NOTES**\n",
        "- code from https://github.com/ageron/handson-ml/blob/master/11_deep_learning.ipynb\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. Clevert, D. A., Unterthiner, T., & Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7Slxf7zuxx9P",
        "colab_type": "code",
        "outputId": "3cfcd7c8-d3e2-4b87-8099-3e6d15a902b0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 284
        }
      },
      "source": [
        "def elu(z, alpha=1):\n",
        "    return np.where(z < 0, alpha * (np.exp(z) - 1), z)\n",
        "\n",
        "plt.plot(z, elu(z), \"b-\", linewidth=2)\n",
        "plt.plot([-5, 5], [0, 0], 'k-')\n",
        "plt.plot([-5, 5], [-1, -1], 'k--')\n",
        "plt.plot([0, 0], [-2.2, 3.2], 'k-')\n",
        "plt.grid(True)\n",
        "plt.title(r\"ELU activation function ($\\alpha=1$)\", fontsize=14)\n",
        "plt.axis([-5, 5, -2.2, 3.2])\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAELCAYAAADECQ0AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd5xU1d3H8c+PpRcBQREFxR6xhCjh\nsbuxa7DFFgsGjWJsAQMaRX2eGAnG2DCiKGpCROzYe2OCQYKCQnBpUgxVijjAwlJ29zx/nFl22R22\nzs6Zu/N9v173xeycmXt/c/bulztnztxrzjlERCS6GoUuQERE6kZBLiIScQpyEZGIU5CLiEScglxE\nJOIU5CIiEacgFxGJOAW5iEjEKcilTsxslJm91YC208jMHjez783MmVlufW+zklrS8poT22pvZsvN\nbO90bK+mzOwlMxsYuo5MZfpmZ/qY2SjgV0maJjnnDk+0d3TO9d7O82PA186568vd3xcY7pxrndKC\nq7fttvj9KB6l7VSy/d7AK0AuMB9Y7ZzbXJ/bTGw3RrnXna7XnNjWvfh97/L63laSbR8LDAIOA3YF\nLnfOjSr3mIOBfwJ7OufWpLvGTNc4dAFZ6COgT7n76j0o6ku6/qjS+Me7D7DMOfdZmra3Xel6zWbW\nErgSOCMd20uiNfA18HRiqcA5N93M5gOXAo+ksbZI0NBK+m1yzn1Xblld3xs1s1PN7FMz+8HMVpvZ\n+2Z2QJl2M7OBZvaNmW0ys8VmdneibRRwHHBdYrjBmVm3kjYze8vM+iXemueU2+6zZvZGdeqoznbK\nrKeZmQ1LbHOjmf3bzI4u0x4zs0fNbKiZrTKzFWZ2n5ltd59PbP9BYPfEtr8ts67h5R9bUk91tlWb\n/q3pa67t6wZOBxwwIUmfHGZmH5tZgZnNNbNjzewCM6vw2Npyzr3jnBvsnHsZKK7koW8AF6Vquw2J\ngjx7tAKGAb3wwwZrgDfNrGmifShwB3A3cCBwPrAo0dYfmAj8HeicWEraSrwEtAVOKrnDzFoDZwHP\nVLOO6mynxF+AC4ErgJ8A04H3zKxzmcdcAhQCRwLXAwMSz9me/sAfgcWJbf+0kseWV9W26tq/UL3X\nXJ1ayjsGmOLKjbOa2U+BT4FxwCHAv4E7gdsSr4Vyjx9sZvlVLMdUUkdVPgd6mVmLOqyjYXLOaUnT\nAozC/4Hll1vuKdP+ViXPj+HHwsvf3xfIr2EtrYAi4Gj8W9uNwG9qse2tNePHlkeXabsUH9TNq1NH\nDbbTCj8cdVmZ9hxgHjCkzHomllvHh8CTVfTLIODbql57uXoq3VZt+7emr7m2rxt4DfhHkvvHAy+U\n+fn0xO9q3HbWsyN+aKqypUUV/Z8P9N1O2yH4dw5712Rfz4ZFY+TpNx7oV+6+dHyYtTdwF/A/wE74\nd2ONgN3xAdEM+LiOm3kG+IeZtXTObcAfGY51zm2sZh3VtTfQhDJDAc65IjObCHQv87j/lHveUmDn\nGmynJirbVnfq3r/Vfc1V1ZJMC2B52TvMbBf8kfrPyty9Gf+7qnA0nqhnNVCfw4QFiX91RF6Ogjz9\nNjjn5tbyuWvxwxfltcMf+VbmLfyQwdXAEvw7gxlA08qeVENvJ9Z7lpl9DJwInJLmOsoOD2xJ0lab\n4cRiwMrd16Tcz6naVm2Un3pW01pWAe3L3Vfy+cnkMvftD8x2zv0r2UrMbDAwuPJSOc0592kVj9me\nHRP/rqzl8xssBXm0zAZONzNzifeaCYcm2pIysw7Aj4BrnXPjEvcdSunvfyawCTgB+GY7q9mMfyu/\nXc65TWb2Ev5IvCPwHf6tfnXrqNZ28MMJm4GjErdJfMh6BPBsFc+tjZX4ceuyfgx8W83np6J/6/M1\nf4UfniurHf4/gKLEttrgx8a/q2Q9jwEvVrGtJbUrEYCDgCXOueVVPjLLKMjTr1nibWtZRc65kqOM\nHcysR7n2uHPuW2AE/sOrh83sCfy46+n4T/LPrGSbP+CPuq4ys0XAbsC9+KNhnHPrzOwh4G4z24Qf\n/ukAHOacG5FYx7f4D5q64ccxVzvnks0weAY/hLAn8Fy5x1RaR3W345xbb2YjgHvMbBWwALgR6AQ8\nWkk/1NYnwDAzOxP/H+bVQFeqGeS17d9y66jP1/x+Yr0dnHPfJ+6bin8XcquZjcH/npYB+5jZvs65\nCv8h1XZoJfGh+D6JHxvhZw31wP/uF5Z56DGJWqW80IP02bTgP7xySZbFVbS/XGYdP8XvzMvxwymT\ngLOrse3j8XN1Nyb+PYUyHyzh/4BuwX8JZjN+1sSfyjx/P/zMig2JmrqVqfmtMo8zfCg54JBa1FHd\n7TTDz35Zjj/a/TeJD0wT7TEq+fCwkn5K9mFnE/zc5VWJ5U4qfthZ6bZq0781fc11fN0TgevK3TcY\n/25kIzAGP/wyAViZ4r+LXJLv96PKPKY5fn8/PPTfcSYu+maniGBmpwIPAd2dc0Wh6ynPzK4DznLO\nnRy6lkykeeQignPuPfy7ji6ha9mOLcANoYvIVDoiFxGJOB2Ri4hEnIJcRCTigkw/7Nixo+vWrVuI\nTW+1fv16WrVqFbSGTKG+8GbPnk1RURHdu5f/omR2ytT9orAQZs2CTZugfXvYa6/632am9MWUKVNW\nOed2Kn9/kCDv1q0bkydPrvqB9SgWi5Gbmxu0hkyhvvByc3OJx+PB981MkYn7xebNcMopPsQPPRQ+\n/RRatqz/7WZKX5jZf5Pdr6EVEYkE5+CGGyAWg86d4fXX0xPiUaAgF5FIePhhGDkSmjeH116DLpk6\nUTIABbmIZLz334cbb/S3//Y36NUrbD2Zps5BbmbNzexzM5tmZnlmdmcqChMRAf/B5oUXQnEx3H47\nXKRrBFWQig87NwHHO+fyzawJ8C8ze9c59+8UrFtEstjq1XDGGbBmDfziF3CnDhOTqnOQO//V0PzE\nj00Si74uKiJ1smULnH8+zJ0LPXrA009DIw0GJ5WS6YeJ8yJPwZ+K8hHn3KQkj+lH4so4nTp1IhaL\npWLTtZafnx+8hkyhvvDi8ThFRUXqi4TQ+8WDD+7LJ5/sRvv2m7n11il88cWmYLWE7osqpfh0lO3w\nF2o9qLLHHXbYYS60cePGhS4hY6gvvOOOO879+Mc/Dl1Gxgi5Xwwf7hw416yZcxMnBitjq0z5GwEm\nuySZmtI3Ks65eCLIT03lekUke3z4IfTv728/9RQcfnjYeqIgFbNWdjKzdonbLYCTgFl1Xa+IZJ85\nc+CCC6CoCG69FS65JHRF0ZCKMfLO+Cun5+D/Y3jROfdWCtYrIlnkhx/8DJV4HM4+G4YMCV1RdKRi\n1sp/gJ+koBYRyVKFhf5IfM4cOOQQGD1aM1RqQl0lIsHdeCN89BHsvDO88Qa0bh26omhRkItIUI89\nBsOHQ9Om8OqrsMceoSuKHgW5iATzySdw/fX+9hNPwJFHhq0nqhTkIhLEN9/Aeef5GSo33wyXXRa6\nouhSkItI2sXjfoZKyUyVoUNDVxRtCnIRSavCQn82w9mz4eCDYcwYyMkJXVW0KchFJK0GDYIPPoCO\nHf0MlTZtQlcUfQpyEUmbJ56Ahx6CJk38DJXA12BvMBTkIpIWsRhce62//fjjcPTRQctpUBTkIlLv\n5s2Dc8/14+MDB8Lll4euqGFRkItIvVqzxs9MWb0afv5zuOee0BU1PApyEak3RUX+GpszZ8KBB8Kz\nz2qGSn1QkItIvbnpJnj3XejQwc9Q2WGH0BU1TApyEakXTz0FDz4IjRvDK6/AXnuFrqjhUpCLSMqN\nHw/XXONvjxgBxx4btp6GTkEuIim1YIGfobJlCwwYAFdeGbqihk9BLiIps3atn6GyahWceirce2/o\nirKDglxEUqKoCC6+GPLy4IAD4Pnn/fi41D8FuYikxC23wNtvw447wptvQtu2oSvKHgpyEamzUaPg\nvvv8EfjYsbD33qEryi4KchGpkwkT4Oqr/e1HHoHc3KDlZCUFuYjU2rffwjnnwObNcMMN0K9f6Iqy\nk4JcRGpl3To480xYuRJOPhkeeCB0RdlLQS4iNVZcDJdeCtOnw/77wwsvaIZKSApyEamxwYP9uVPa\nt/czVNq1C11RdlOQi0iNPP20PxVtTg68/DLsu2/oikRBLiLVNnEiXHWVv/3ww3D88WHrEU9BLiLV\nsnAhnH22n6Fy3XWlJ8WS8BTkIlKl/Hw/Q2XFCjjhBH96WskcCnIRqVRxMfTpA9Om+fHwl16CJk1C\nVyVlKchFpFJ33AGvveZnprz5pp+pIpmlzkFuZl3NbJyZzTCzPDPrn4rCRCS8MWNg6FA/Q+XFF/2c\ncck8qTgiLwQGOue6A4cD15lZ9xSsV0QCmjGjDb/+tb89bBicdFLYemT76hzkzrllzrkvE7fXATOB\n3eq6XhEJZ9EiuP32g9m0CX7zGz9LRTJXSsfIzawb8BNgUirXKyLps349nHUW/PBDU372M/jrX8Es\ndFVSmZSdHcHMWgNjgQHOubVJ2vsB/QA6depELBZL1aZrJT8/P3gNmUJ94cXjcYqKirK6L4qL4c47\nD+Srr3aic+f19O//FRMmFIYuK7hM/xtJSZCbWRN8iI9xzr2S7DHOuZHASICePXu63MAnLY7FYoSu\nIVOoL7x27doRj8ezui/+939h/HjYYQe4++48zjrr6NAlZYRM/xupc5CbmQFPATOdczqRpUhEPf88\n3HUXNGrkz2bYvPmG0CVJNaVijPwooA9wvJlNTSynp2C9IpImn38Ol1/ubz/wAJx6ath6pGbqfETu\nnPsXoI9CRCJqyRJ/DpWNG/0JsX7729AVSU3pm50iWWzDBj9DZdkyOO44GD5cM1SiSEEukqWc88Mp\nU6bAXnv5c4s3bRq6KqkNBblIlvrjH/3X7tu08Vf76dgxdEVSWwpykSz00kvwhz/4GSrPPw8HHhi6\nIqkLBblIlpkyBX71K3/73nvhdM0xizwFuUgWWbrUXyCioACuuAJuvDF0RZIKCnKRLFFQ4KcZLl0K\nxxwDI0ZohkpDoSAXyQLO+SPwL76Abt1g7FjNUGlIFOQiWeBPf/IfarZu7a/ys9NOoSuSVFKQizRw\nY8f6y7WZwXPPwUEHha5IUk1BLtKAffUVXHaZv33PPdC7d9h6pH4oyEUaqGXL/AyVDRv8dMNBg0JX\nJPVFQS7SAG3cCOecA4sXw1FHweOPa4ZKQ6YgF2lgnIMrr4RJk2CPPeCVV6BZs9BVSX1SkIs0MH/+\nM4wZA61a+XOo7Lxz6IqkvinIRRqQ116DwYP9MMqYMXDIIaErknRQkIs0ENOmwaWX+ttDh/rzjEt2\nUJCLNADLl8MZZ8D69dCnD/z+96ErknRSkItEXMkMlUWL4PDDYeRIzVDJNgpykQhzDvr1g4kToWtX\nP0bevHnoqiTdFOQiEXbvvTB6NLRs6WeodOoUuiIJQUEuElFvvAG33OJvP/MM9OgRth4JR0EuEkHT\np8Mll/ihlSFD/Bi5ZC8FuUjErFjhZ6jk58PFF/t545LdFOQiEbJpE/ziF/Df/0KvXvDkk5qhIgpy\nkchwDn7zG5gwAbp08TNUWrQIXZVkAgW5SETcfz+MGuXD+/XXoXPn0BVJplCQi0TA22/DzTf726NH\nw6GHhq1HMouCXCTD5eXBRRf5oZU//hHOPTd0RZJpFOQiGWzVKj9DZd06uPBCuP320BVJJlKQi2So\nzZv90feCBdCzJ/z975qhIsmlJMjN7G9mtsLMvk7F+kSynXNw7bUwfjzsuqv/cFMzVGR7UnVEPgo4\nNUXrEsl6w4bBU0+VzlDZddfQFUkmS0mQO+fGA6tTsS6RbPfuu6VXvB81yg+riFRGY+QiGWTGDPjl\nL6G4GP7v/+CCC0JXJFHQOF0bMrN+QD+ATp06EYvF0rXppPLz84PXkCnUF148HqeoqChYX6xZ05hr\nrz2MtWtbcNxxKzj22BmE/LVovyiV6X2RtiB3zo0ERgL07NnT5ebmpmvTScViMULXkCnUF167du2I\nx+NB+mLzZjjlFFi61H/Z5513dqZly53TXkdZ2i9KZXpfaGhFJDDn4IYbIBbzX7t//XV/oQiR6krV\n9MPngInA/ma22Mx+nYr1imSDhx/219ls3tyfCKtLl9AVSdSkZGjFOXdRKtYjkm3efx9uvNHf/tvf\n/KlpRWpKQysigcya5b92X1zsv3p/kQ6HpJYU5CIBrF7tz6GyZo2/UMSdd4auSKJMQS6SZlu2wPnn\nw9y5/oLJTz8NjfSXKHWg3Uckzfr3h08+gU6d4I03oFWr0BVJ1CnIRdLokUdgxAho1szPUOnaNXRF\n0hAoyEXS5MMP/dE4+BNiHX542Hqk4VCQi6TBnDn+vClFRXDrrXDJJaErkoZEQS5Sz374wc9Qicfh\n7LNhyJDQFUlDoyAXqUdbtvgj8Tlz4JBD/IWTNUNFUk27lEg9+t3v4KOPYOed/QyV1q1DVyQNkYJc\npJ489hgMHw5Nm8Krr8Iee4SuSBoqBblIPfjkE7j+en/7iSfgyCPD1iMNm4JcJMW++QbOO8/PULn5\nZrjsstAVSUOnIBdJoXjcz1ApmakydGjoiiQbKMhFUqSw0J/NcPZsOPhgGDMGcnJCVyXZQEEukiID\nB8IHH8BOO/kZKm3ahK5IsoWCXCQFRo6Ev/4VmjSBV16Bbt1CVyTZREEuUkfjxsF11/nbI0fC0UeH\nrUeyj4JcpA7mzvUzVAoLYdAg6Ns3dEWSjRTkIrW0Zg2ceaa/2k/v3vDnP4euSLKVglykFgoL4Ze/\nhJkz4cADNUNFwlKQi9TCTTfBe+9Bx47w5puwww6hK5JspiAXqaEnn4Rhw0pnqOy5Z+iKJNspyEVq\n4J//hGuu8bcfewyOOSZsPSKgIBeptvnz4dxz/fj4734HV1wRuiIRT0EuUg1r1/pzp3z/PZx2Gvzl\nL6ErEimlIBepQlERXHQRzJgBBxwAzz2nGSqSWRTkIlW4+WZ45x3YcUc/Q6Vt29AViWxLQS5Siaee\nggcegMaNYexY2Hvv0BWJVKQgF9mO8eNLZ6g8+ijk5gYtR2S7FOQiSSxY4GeobNkC/fvDVVeFrkhk\n+xTkIuWUzFBZtQpOOQXuuy90RSKVS0mQm9mpZjbbzOaa2S2pWKdICM7BxRdDXh786Efwwgt+fFwk\nk9V5FzWzHOAR4CRgMfCFmb3hnJtR13WLpNuyZS34z380Q0WiJRXHGr2Auc65+QBm9jxwFrDdIJ89\neza5gT85isfjtGvXLmgNmUJ94X3++VQKCgBy6doVrrwydEVhab8olel9kYog3w1YVObnxcD/lH+Q\nmfUD+gE0adKEeDyegk3XXlFRUfAaMoX6Atavb5wIcejSZQOwmSzvEu0XZWR6X6Rt9M85NxIYCdCz\nZ083efLkdG06qVgsFvxdQabI9r7Iyyu5PFsuHTtuYtGiiaFLygjZvl+UlSl9YWZJ70/Fh51LgK5l\nfu6SuE8k4y1eDKeeCvE4dOgAu+5aELokkRpLxRH5F8C+ZrYnPsB/CVycgvWK1KsffvAhvngxHHUU\nNGrkpx6KRE2dj8idc4XA9cD7wEzgRedcXl3XK1Kf8vP9XPG8PH8irDfe8EEuEkUpGSN3zr0DvJOK\ndYnUt/Xr4ec/hwkToEsXf8m2HXcMXZVI7ekYRLLK+vX+ivfjx8Nuu8G4cbD77qGrEqkbBblkjZLh\nlFgMOnf2Ib7PPqGrEqk7fflYssKqVX445fPPYZddfIjvu2/oqkRSQ0fk0uAtXOjniX/+Oeyxh7+A\n8v77h65KJHUU5NKg5eX5qYWzZ8PBB8Nnn8F++4WuSiS1FOTSYL31FhxxhJ8nfvTR/gPOXXcNXZVI\n6inIpcFxzl/l/swzYd06uPBC+OADyOBzHonUiYJcGpT8fOjTB37/ex/oQ4b4q963aBG6MpH6o1kr\n0mBMnw4XXACzZkGrVjB6NJxzTuiqROqfjsgl8pyDJ5+EXr18iHfv7meoKMQlWyjIJdK++84H9lVX\nwcaNcMUV8MUXPsxFsoWCXCLrhRfgoIPg9ddhhx3g6afhqaegZcvQlYmkl8bIJXIWLoT+/eG11/zP\nJ53kh1Z0zhTJVjoil8jYssVPKzzgAB/irVvDY4/B++8rxCW76YhcMp5z8M47cNNNMHOmv+/88+GB\nB/xpaEWynYJcMtqXX8KgQf4kVwB77w3Dh/sr+4iIp6EVyUjTp/uj7sMO8yHevr0/As/LU4iLlKcj\ncskoU6fCn/4EL7/sf27WDK6/Hm67zYe5iFSkIJfgiorg7bfhwQf9RR/AB/jVV/uv2utEVyKVU5BL\nMOvWwahR8NBDMG+ev69NG7jySj8urgAXqR4FuaSVc/50sn//ux8+Wb/e39+tG/z2t/DrX/sv94hI\n9SnIJS0WLIBnnvFH4PPnl95/zDH+yz1nnw05OcHKE4k0BbnUmzlz/FH32LF+GmGJLl3gssugb19d\nN1MkFRTkkjKFhTBpErz3nv/m5ddfl7a1bu2vYN+3L5xwgo6+RVJJQS51snAhfPihD+8PP4Q1a0rb\n2rb1V+k591w4+WRd3EGkvijIpdqc8xcx/vRT/4Hl+PE+yMvad1//hZ3TTvNH3k2bhqlVJJsoyCUp\n5/xFiydPLl2mTIHvv9/2ce3awbHH+vA+5RTYa68w9YpkMwW5sHFjI7780n/9fcYMmDbNh/aKFRUf\n26mTD+6S5aCDoJFO9CASlII8S2zZAosW+al/8+fD3Ln+TIJ5efDtt8fgXMXntG8PPXtuu3TtCmbp\nr19Etk9B3kDk58OSJbB0qV8WLiwN7fnzfYgXFSV/bk6OY//9je7d4cAD/XLYYbDnngptkShQkGeo\n4mKIx/2Y9KpV2y4rV8KyZT6wS8J73brK12fmj6b32ssve+7pL9Bw4IGwZMmnnHjicel5YSKScnUK\ncjM7H/gDcADQyzk3ORVFRV1RERQUwIYNPmDXrvXT8qr6d82a0uD+/nsf5tXVvLk/N8luu/l/u3Qp\nDe299oI99vAnokpm+fIk4yoiEhl1PSL/GvgF8HgKaqmR4mIfmCVLYWHFn7dsgc2bky+TJ+/IDz9U\n/piSpaCgNJg3bKj69ubNqXmNbdtCx44Vlw4doHPnbYO7XTsNg4hkqzoFuXNuJoDVMEG++mo2rVvn\n4hxbP2Rr2fICWre+li1bNrBq1elb20qWnJy+mPWlsHAVxcXnJVnrNcCFwCKgT5L2gcAZwGzg6iTt\ntwMnAlOBAUnahwJHAp8Bg5O0DwN6AB8BQ2jUyM/maNzYf4vxgAMeZ5dd9mfdujf55pv7yckpbWvc\nGAYNGs0++3Tliy9e4JVXRtCkybbBPGrUy3Ts2JFRo0YxatSoClt/5513aNmyJY8++igvvvhihfZY\n4vyw9913H2+99dY2bQUFBUyaNAmAu+66i48//nib9g4dOjB27FgAbr31ViZOnLhNe5cuXXjmmWcA\nGDBgAFOnTt2mfb/99mPkyJEA9OvXjzlz5mzT3qNHD4YNGwbApZdeyuLFi7dpP+KII7j77rsBOPfc\nc/m+3BzIE044gTvuuAOA0047jYKCgm3ae/fuzaBBgwDIzc2lvAsuuIBrr72W4uJi5s6dW+Exffv2\npW/fvqxatYrzzqu4711zzTVceOGFLFq0iD59Ku57AwcO5IwzzmD27NlcfXXFfe/222/nxBNPZOrU\nqQwYUHHfGzp0KEceeSSfffYZgwdX3PeGDRtGjx49+OijjxgyZEiF9scff5z999+fN998k/vvv79C\n++jRo+natSsvvPACI0aM2Hp/PB6nXbt2vPxy/e17LVq04N133wWye9/bsGEDp59+eoX2qva9Emkb\nIzezfkA//1PrrWe9K1FQUHGOclnbG2bwYedo0qSIpk23AFvYuNEBDjMfpmaO9u0LaN9+DYWFa1m6\ntBBwiTbfvs8+39O581Ly85fz9debMHOJNmjUyHHMMQvp1q09K1fOZ9y49TRq5Lauu1EjxxVXTOVH\nP8onL28azz8fr1DnDTdMYvfdl/HZZ9OJxyu2t2kzEefmsXZtHhs2VGyfMGECbdu2ZdasWUmfP378\neJo3b86cOXOStpf8Mc2bN69Ce05Oztb2BQsWVGgvLi7e2r5w4cIK7U2aNNnavnjx4grtS5cu3dq+\ndOnSCu2LFy/e2r58+fIK7QsXLtzavnLlStauXbtN+4IFC7a2r169mk2bNm3TPm/evK3tyfpmzpw5\nxGIx4vE4zrkKj5k1axaxWIw1a9YkfX5eXh6xWIwVK1YkbZ8+fTpt2rRJ2ncA06ZNo3HjxsydOzdp\n+5dffsnmzZv5+uuvk7ZPnjyZeDzOtGnTkrZPmjSJZcuWMX168n1v4sSJzJs3j7y8vG3ai4qKiMfj\n9brvFRQURGLfy8/Pr9d9b+PGjUnbq9r3SphLNu+s7APMPgJ2SdJ0m3Pu9cRjYsCg6o6Rd+/e0z37\n7GRycqh0KTliTbbUde5yLBZL+j9kNlJfeLm5ucTj8QpHddlK+0WpTOkLM5vinOtZ/v4qj8idcyem\nupiWLaFHj1SvVUQkO+k7eSIiEVenIDezc8xsMXAE8LaZvZ+askREpLrqOmvlVeDVFNUiIiK1oKEV\nEZGIU5CLiEScglxEJOIU5CIiEacgFxGJOAW5iEjEKchFRCJOQS4iEnEKchGRiFOQi4hEnIJcRCTi\nFOQiIhGnIBcRiTgFuYhIxCnIRUQiTkEuIhJxCnIRkYhTkIuIRJyCXEQk4hTkIiIRpyAXEYk4BbmI\nSMQpyEVEIk5BLiIScQpyEZGIU5CLiEScglxEJOIU5CIiEacgFxGJOAW5iEjEKchFRCKuTkFuZvea\n2Swz+4+ZvWpm7VJVmIiIVE9dj8g/BA5yzh0CzAFurXtJIiJSE3UKcufcB865wsSP/wa61L0kERGp\niVSOkV8BvJvC9YmISDU0ruoBZvYRsEuSptucc68nHnMbUAiMqWQ9/YB+AJ06dSIWi9Wm3pTJz88P\nXkOmUF948XicoqIi9UWC9otSmd4X5pyr2wrM+gJXAyc45zZU5zk9e/Z0kydPrtN26yoWi5Gbmxu0\nhkyhvvByc3OJx+NMnTo1dCkZQftFqUzpCzOb4pzrWf7+Ko/Iq1jpqcDNwHHVDXEREUmtuo6RDwfa\nAB+a2VQzeywFNYmISA3U6QY7FMAAAAKVSURBVIjcObdPqgoREZHa0Tc7RUQiTkEuIhJxCnIRkYir\n8/TDWm3UbCXw37RveFsdgVWBa8gU6otS6otS6otSmdIXezjndip/Z5AgzwRmNjnZfMxspL4opb4o\npb4olel9oaEVEZGIU5CLiERcNgf5yNAFZBD1RSn1RSn1RamM7ousHSMXEWkosvmIXESkQVCQA2Y2\n0MycmXUMXUsoumyfPwmcmc02s7lmdkvoekIxs65mNs7MZphZnpn1D11TaGaWY2ZfmdlboWtJJuuD\n3My6AicDC0PXElhWX7bPzHKAR4DTgO7ARWbWPWxVwRQCA51z3YHDgeuyuC9K9Admhi5ie7I+yIEH\n8afizeoPC3TZPnoBc51z851zm4HngbMC1xSEc26Zc+7LxO11+ADbLWxV4ZhZF+DnwJOha9merA5y\nMzsLWOKcmxa6lgyTjZft2w1YVObnxWRxeJUws27AT4BJYSsJahj+YK84dCHbU6fT2EZBZZeqAwbj\nh1WyQqou2yfZwcxaA2OBAc65taHrCcHMegMrnHNTzCw3dD3b0+CD3Dl3YrL7zexgYE9gmpmBH0r4\n0sx6Oee+S2OJabO9viiRuGxfb/xl+7JtqGkJ0LXMz10S92UlM2uCD/ExzrlXQtcT0FHAmWZ2OtAc\n2MHMnnHOXRq4rm1oHnmCmX0L9HTOZcKJcdIucdm+B/CX7VsZup50M7PG+A95T8AH+BfAxc65vKCF\nBWD+yOYfwGrn3IDQ9WSKxBH5IOdc79C1lJfVY+Syjay+bF/ig97rgffxH+69mI0hnnAU0Ac4PrEv\nTE0ckUqG0hG5iEjE6YhcRCTiFOQiIhGnIBcRiTgFuYhIxCnIRUQiTkEuIhJxCnIRkYhTkIuIRNz/\nAzT4EGRg4ufNAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gg_Zzluex0L_",
        "colab_type": "text"
      },
      "source": [
        "A further activation function is *scaled exponential linear units (SELU)*<sup>1</sup> which, when combined with LeCun initialization, will self-normalize to prevent the vanishing/exploding gradients problem. The default SELU hyperparameters (scale and alpha) are tuned so that the mean output of each neuron remains close to 0, and the standard deviation remains close to 1 provided the inputs are standardized with mean 0 and standard deviation 1 too<sup>2</sup>.\n",
        "\n",
        "However, as the self-normalization is easily broken, it cannot be used with<sup>2</sup>:\n",
        "- ℓ1 or ℓ2 regularization\n",
        "- regular dropout (but can with Alpha Dropout: tf.contrib.nn.alpha_dropout())\n",
        "- max-norm\n",
        "- skip connections or other non-sequential topologies\n",
        "\n",
        "**NOTES**\n",
        "- code from https://github.com/ageron/handson-ml/blob/master/11_deep_learning.ipynb\n",
        "\n",
        "---\n",
        "1. Klambauer, G., Unterthiner, T., Mayr, A., & Hochreiter, S. (2017). Self-normalizing neural networks. In Advances in neural information processing systems (pp. 971-980).\n",
        "2. https://github.com/ageron/handson-ml/blob/master/11_deep_learning.ipynb"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YSfJd5oix4Ug",
        "colab_type": "code",
        "outputId": "1f990c3c-b793-4455-b9b0-6f02207f85ad",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "source": [
        "from scipy.special import erfc\n",
        "\n",
        "# alpha and scale to self normalize with mean 0 and standard deviation 1\n",
        "# (see equation 14 in the paper):\n",
        "alpha_0_1 = -np.sqrt(2 / np.pi) / (erfc(1/np.sqrt(2)) * np.exp(1/2) - 1)\n",
        "scale_0_1 = (1 - erfc(1 / np.sqrt(2)) * np.sqrt(np.e)) * np.sqrt(2 * np.pi) * (2 * erfc(np.sqrt(2))*np.e**2 + np.pi*erfc(1/np.sqrt(2))**2*np.e - 2*(2+np.pi)*erfc(1/np.sqrt(2))*np.sqrt(np.e)+np.pi+2)**(-1/2)\n",
        "\n",
        "def selu(z, scale=scale_0_1, alpha=alpha_0_1):\n",
        "    return scale * elu(z, alpha)\n",
        "\n",
        "plt.plot(z, selu(z), \"b-\", linewidth=2)\n",
        "plt.plot([-5, 5], [0, 0], 'k-')\n",
        "plt.plot([-5, 5], [-1.758, -1.758], 'k--')\n",
        "plt.plot([0, 0], [-2.2, 3.2], 'k-')\n",
        "plt.grid(True)\n",
        "plt.title(r\"SELU activation function\", fontsize=14)\n",
        "plt.axis([-5, 5, -2.2, 3.2])\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEJCAYAAACJwawLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfZxWc/7H8denWymJ7lBRi9CiKPdL\nQ9nkXkVSrYiitYoIlTZ3YYnYVVu/VpFQym2rRTRrLUrRJtlJYiuioqua7qZmvr8/vteYaW6qaa65\nvtfN+/l4nEdnrnPmnM/17Zr3nDnne87XnHOIiEjyqhS6ABERKR8FuYhIklOQi4gkOQW5iEiSU5CL\niCQ5BbmISJJTkEtSMrNvzey2OOxnuJktisN+KpnZWDP7ycycmWVU9D53U89EM5sRsgbZcwryJGdm\n9c1sdDTYtpnZj2b2rpmdW2idzGg4FJ1eLLSOM7Mupeyjl5lll7Ks1O+LhV0E6UnA6Bjup2n0vbQp\nsuhRoG2s9rML5wPXABcBBwMfxmGfmFlG9H3XK7KoP9AjHjVI+VUJXYCU23RgX6A3sBRogA+eukXW\nmwAMLvLalgqvroI459bEaT/ZQIm/xGLsCGCVcy4uAb47zrn1oWuQMnDOaUrSCagDOKD9btbLBP6y\nm3Uc0KWUZb2A7LJ+X3T5ecC/gHXAz8BbwDFF1jkEmAz8BGwGFgBnR/friky9ot/zLXBbdP55YHqR\nbVYCVgC37kkdJewnM/r6cGBRke3eHd32NuBz4JJCy5tGv78z8E70/SwGzt1FG00ssu9vS/t/i647\no8j/7WhgBLAWWI3/K6JSoXWqRZf/L1rzMuDmQrUWniaWsp/qwCjgR2Ar8DHwm0LLM6Lf3w6YE33f\n84ATQ/+cpMOkUyvJLf9o8WIz2yd0MaWoiQ+Ak/E/7OuBN8ysGoCZ1QT+iQ+VS4HjgHuj3zsFGAlk\n4U83HBx9rajngAvMbP9Cr7WNrv/CntQRfR184B8MdCrl/fQHbgfuiNb6CvCymbUqst4DwJNAS+AT\n4EUzq7WLbd4LrIzu+6RS1itNd2AHcDpwEzAA6Fpo+TPA74BbgWPwf71F8L+MOkfX+XV03/1L2cef\notu8FjgB/wvsH2Z2cJH1HgTuBE7E/2KebGZWxvcjZRX6N4mm8k34H8Sf8UdJH+GPxk4psk4mkENB\n8OdP/QqtUyFH5CWsXxPIJXo0B1wPbATqlbL+cAodERd6/VsKjsir4I8UexdaPh54uwx1NI2+lza7\n2j/wHTCshPZ9rsh2+hZa3ij62m92Uc9tRI/Ei2x3T47IPyqyzjvA+Oj8kdF9n1fKfjOiy+uVtp9o\nW+UAvyu0vDLwNXB/ke10KLTOGdHXGof+OUn1SUfkSc45Nx1/auIiYCb+qOxjMyt6PnwK0KrINLmi\n6zOzw83seTP72sw24AO3EnBodJUTgIXOubV7uw/n3A78++se3Wd1/C+458pQx568l9r4tv53kUUf\nAC2KvLaw0Pz30X8b7Om+ymhhka+/L7SvE4A8YHY5tn84UJVC79s5l4s/cAj5viVKFztTgHNuK/4o\n7B3gXjMbDww3s0edcznR1dY755bu5S42ADXMrKpzbnv+i2ZWJ3/bu/jeGfhTBn3xR7M78OeMq+3i\ne/bGc8BHZtYIOCW6/ZfjWEfRx4j+0k7OORc9u1DWA6c8oOhpiaolrLe9yNduL/a1t0p934WW6YCx\ngqmBU9Ni/C/pWJ03z8J/Vk4o8vqJhZYXY2Z1gaOBEc65Wc65L4H92PkA4jPg+BK6v+XLwf8Zv0vO\nubn4Xjvd8Efmrznf42RP68j/hVfqvpxzG/BHmWcUWfQbfJvH2hr8eevCWpZxGwvw/3dnl7J8t+8b\nfwolh0Lv28wqA6dRMe9bykhH5EksGlAvAU/j/6TdCLQBBgHvRoMn375mdlCRTeQ4534u9HXTEi7a\nLXPOfWFmbwPjzexW/A92c+AJYKpzbnkpJa7D96S43sxW4M8VP4I/Gs73PP7i2Gtmdif+aPlYYKNz\nbjb+XPhhZnYisDz6+rZS9jcZuA5/nrrwxco9qWM1vjtmBzP7FtjqSu6C9wj+r56vgPn4vtZnUvBL\nLZbeA0aZ2cX4X5Z9gSb4NtkjzrklZjYV/3/XH/gUaAw0dc5NwvdkcfiLxW8AW/J/ARbaxiYzGwM8\nbGZrgW+AW4CGxLAvv5RD6JP0mvZ+wncJG4HvFbEO3+XrK+Ax4MBC62VSvJuZAz4otE5Jyx1wYXR5\nHXxwL43uZwnwMFBrNzWeAyzCX4xdBHTAX2jtVWidxvhz3JHotj8DMgq9x2nR91di98NC2/lVdJ0f\ngSp7Ucd1+F8WuexZ98McfO+NSwstb0rJF013102zpIudVYGn8L+E1gL3UPLFzt1dEK2O73XyHb77\n4dfATYWW3w2swp/KmbiLbeR3P9xG6d0P6+2uLTTFfrJog4uISJLSOXIRkSSnIBcRSXIKchGRJKcg\nFxFJckG6H9arV881bdo0xK5/sWnTJmrWrBm0hkShtvCysrLIzc2lRYuiNyump0T4XGzbBl9+Cbm5\ncMghcHDRXvVxkghtATB//vy1zrn6RV8PEuRNmzZl3rx5IXb9i8zMTDIyMoLWkCjUFl5GRgaRSCT4\nZzNRhP5cZGfDaaf5EL/0Upg+HSoFOocQui3ymdn/Snpdp1ZEJOE4B9ddB4sWwVFHwTPPhAvxZKCm\nEZGE89hjMGUK1KoFr7wCtWuHriixKchFJKG89x4MGuTnn30WjjkmbD3JoNxBbmb7mNlcM/uPmX1h\nZvfEojARST/Ll0PXrpCXB3fdBZddFrqi5BCLi53bgHOcc9lmVhX4wMxmOuc+jsG2RSRNbN0KnTvD\n2rXQoQPcd1/oipJHuYPc+Ye15D8trWp00gNcRGSPOQf9+sG8edCsGTz/PFTe7cOLJV9Muh9Gn008\nHz8S+FPOuTklrNMH6APQsGFDMjMzY7HrvZadnR28hkShtvAikQi5ublqi6h4fi5ef/0QJkxoTvXq\nuQwe/BkLF2bv/pviKOF/RmL5KEX8o05nA8fuar3WrVu70GbPnh26hIShtvDatm3rWrZsGbqMhBGv\nz8WHHzpXtapz4NykSXHZZZklys8IMM+VkKkx7bXinItEg/y8WG5XRFLTDz9Aly6wfTvcfDP06BG6\nouQUi14r9fPHbjSzGsC5wH/Lu10RSW3bt8Pll8P338OZZ8Kjj4auKHnF4hz5wcAz0fPklfBDf82I\nwXZFJIUNHAgffOCfoTJ1KlQtaVhp2SOx6LWykOKD8oqIlGrSJPjzn314T58OBxUdTVbKRHd2ikhc\nffYZ9Onj5//yFzj11LD1pAIFuYjEzU8/QadO/uaf3r3h+utDV5QaFOQiEhe5udCtG3z7LZx0kj8a\nNwtdVWpQkItIXAwdCu+8A/Xr+/Pi++wTuqLUoSAXkQo3fTo89JC/7X7qVGjSJHRFqUVBLiIVavFi\n6NXLzz/yCCTAQDspR0EuIhVm/Xr/KNrsbLjyShgwIHRFqUlBLiIVIi8Prr4aliyB446D8eN1cbOi\nKMhFpEKMGAGvvQZ16vjh2hJgEPqUpSAXkZibOROGDfNH4M8/D4cfHrqi1BaT55GLiOT7+mu46io/\nWMS990LHjqErSn06IheRmNm0yV/cjETg4othyJDQFaUHBbmIxIRz/pb7zz+H5s3h2WehkhImLtTM\nIhITo0bBCy9ArVr+4ub++4euKH0oyEWk3DIz4fbb/fzEidCiRchq0o+CXETKZcUKuOIK/1CsO+6A\nzp1DV5R+FOQiste2bvXBvWYNnHsuPPBA6IrSk4JcRPbaH/4An3wChx3mz49Xrhy6ovSkIBeRvTJu\nnL/tfp99/MXNunVDV5S+FOQiUmYffww33eTnx42DEzRqb1AKchEpkx9/hC5dYPt2H+Y9e4auSBTk\nIrLHtm/3PVS++w5+8xsYOTJ0RQIKchEpg9tvh/ffh4MPhpdegmrVQlckoCAXkT00eTI88QRUreqH\nbjvooNAVST4FuYjs1oIF/jkqAE8+CaedFrYe2ZmCXER26eefoVMn2LIFrrkG+vYNXZEUpSAXkVLl\n5vpni3/zDbRpA6NHa7i2RKQgF5FSDRsGb70F9er58+L77BO6IimJglxESvSvf9VjxAj/TPEpU+DQ\nQ0NXJKVRkItIMf/9Lzz00NEAPPwwnHNO4IJklxTkIrKTDRvg0kth8+YqdO0KAweGrkh2R0EuIr/I\ny4Orr4asLGjWLJu//U0XN5NBuYPczJqY2WwzW2xmX5hZ/1gUJiLx99BD8Oqrfpi2++77gpo1Q1ck\ne6JKDLaxAxjonPvUzPYD5pvZO865xTHYtojEyVtvwdChfn7yZKhZc0vYgmSPlfuI3Dm3yjn3aXR+\nI/Al0Ki82xWR+Fm2DLp1A+dg+HC44ILQFUlZxOKI/Bdm1hQ4AZhTwrI+QB+Ahg0bkpmZGctdl1l2\ndnbwGhKF2sKLRCLk5uamXVts3VqJm246kXXranH66Ws588xFZGbqc1FYwreFcy4mE1ALmA902t26\nrVu3dqHNnj07dAkJQ23htW3b1rVs2TJ0GXGVl+dc9+7OgXNHHulcJFKwTJ+LAonSFsA8V0KmxqTX\niplVBaYDk51zL8dimyJS8Z58Mv98uB+ubf/9Q1ckeyMWvVYM+BvwpXPusfKXJCLx8M9/FvQRnzAB\nfv3rsPXI3ovFEfkZQE/gHDNbEJ3Oj8F2RaSCrFzpR/rJzfWDRVx+eeiKpDzKfbHTOfcBoFsGRJLE\ntm1+zM3Vq6FdOxgxInRFUl66s1Mkzdx8M8yZ4x+C9eKLUCWmfdckBAW5SBoZPx7GjYPq1eHll/3j\naSX5KchF0sTcufD73/v5sWOhdeuw9UjsKMhF0sDq1dC5M+TkQL9+/sFYkjoU5CIpbscO30Nl5Uo4\n/XR4/PHQFUmsKchFUtygQb7P+EEHwUsvQbVqoSuSWFOQi6SwF17wR+BVqsC0aXDIIaErkoqgIBdJ\nUQsXQu/efv6JJ+CMM8LWIxVHQS6Sgtatg8sugy1b/IXNG28MXZFUJAW5SIrJzYXu3f0zxk88EcaM\n0XBtqU5BLpJihg+HmTOhbl1/00+NGqErkoqmIBdJIa+9BvffD5Uq+dvvDzssdEUSDwpykRSRlQU9\ne/r5Bx+E9u3D1iPxoyAXSQEbN/qLmxs3+kfS3n576IoknhTkIknOObjmGvjyS2jRAp5+Whc3042C\nXCTJPfwwTJ8OtWv74dpq1QpdkcSbglwkib39NgwZ4uefew6aNw9bj4ShIBdJUt98A926QV4eDBsG\nF10UuiIJRUEukoQ2b4ZOneDnn+H88+GPfwxdkYSkIBdJMs7BDTfAggVw+OH+lEol/SSnNf33iySZ\np56CSZNg333h1VfhgANCVyShKchFksi//gW33OLnn34ajj02bD2SGBTkIkniu+/8zT47dsDAgdC1\na+iKJFEoyEWSwLZt0KUL/PgjnH02PPRQ6IokkSjIRZLAgAHw8cfQpAlMmeJH/BHJpyAXSXBPPw1/\n/StUr+4fS1u/fuiKJNEoyEUS2Lx50K+fnx8zBtq0CVuPJCYFuUiCWrPG3/SzbZvvN37NNaErkkSl\nIBdJQDt2wJVXwooVcOqpfvBkkdIoyEUS0J13wnvvQcOG/smG1aqFrkgSmYJcJMFMmQIjR/qeKS+9\nBIccEroiSXQxCXIze9rMVpvZolhsTyRdff45XHutn3/sMTjzzLD1SHKI1RH5ROC8GG1LJC1FIn64\nts2b/dibN90UuiJJFjEJcufc+8DPsdiWSDrKy4Pu3eHrr6FVKxg7VsO1yZ6L2/1hZtYH6APQsGFD\nMjMz47XrEmVnZwevIVGoLbxIJEJubm6QtpgwoSlvvtmU2rW3c8cd85kzZ2vcayhKn4sCid4WcQty\n59w4YBxAmzZtXEZGRrx2XaLMzExC15Ao1BZenTp1iEQicW+LN96AZ5/1zxSfNq0q5557alz3Xxp9\nLgokeluo14pIQEuWQI8efv6BB+Dcc8PWI8lJQS4SSHa2v7i5YQN07gx33BG6IklWsep++ALwEXCU\nma00s96x2K5IqnLO33K/eDEccwxMmKCLm7L3YnKO3DnXLRbbEUkXjz4K06ZB7drwyiuw336hK5Jk\nplMrInE2a5a/BR/8Rc6jjgpbjyQ/BblIHH37rX8YVl4eDB0Kl1wSuiJJBQpykTjZssVf1PzpJ+jY\nEYYPD12RpAoFuUgcOAc33giffgq/+hVMngyVK4euSlKFglwkDsaMgWeegX339Rc3DzggdEWSShTk\nIhXs3/+G/v39/PjxcPzxYeuR1KMgF6lAq1ZBly5+xJ9bboFu6qgrFUBBLlJBcnJ8iP/wA2RkwJ/+\nFLoiSVUKcpEKcsst8OGH0LixH/WnStweUSfpRkEuUgEmToTRo/1Ym9OnQ4MGoSuSVKYgF4mx+fPh\nhhv8/FNPwcknh61HUp+CXCSG1q6FTp1g2zbo0weuuy50RZIOFOQiMbJjh7/9fvlyOOUUePLJ0BVJ\nulCQi8TIkCHw7rv+fPi0aVC9euiKJF0oyEVi4KWXfPfCypX9fOPGoSuSdKIgFymnRYv8IBEAI0fC\nWWeFrUfSj4JcpBwiEX9xc9Mm6N4dbr45dEWSjhTkInspLw969oSvvoKWLWHcOA3XJmEoyEX20v33\nw4wZ/kmGL7/sn2woEoKCXGQv/P3vfmAIM3jhBf+McZFQ9PQHkTL66it/Ptw5eOAB6NAhdEWS7nRE\nLlIG2dn+4ub69XDZZXDXXaErElGQi+wx56B3b9/d8Oij/YOxdHFTEoGCXGQPPfYYTJ0K++3nh2ur\nXTt0RSKeglxkD7z3Hgwa5OefecYfkYskCgW5yG4sXw5du/p+44MH+3PjIolEQS6yC1u3QufO/vG0\nHTrAvfeGrkikOAW5SCmcg379YN48aNYMnn/ePxRLJNEoyEVKMXYsTJgANWr4i5sHHhi6IpGSKchF\nSvDRRwUPwPq///PPUhFJVApykSJ++MGfF9++Hfr393dxiiSymAS5mZ1nZllmttTM7ozFNkVCcA4u\nvxxWrfLPFX/kkdAViexeuZ+1YmaVgaeAc4GVwCdm9rpzbnF5ty0Sb6tW1WDhQmjUyN/8U7Vq6IpE\ndi8WD806GVjqnFsGYGYvApcApQZ5VlYWGRkZMdj13otEItSpUydoDYlCbeF9+ukCNm4EyKBBA993\nPJ3pc1Eg0dsiFkHeCFhR6OuVwClFVzKzPkAfgKpVqxKJRGKw672Xm5sbvIZEobbwNm92gHHggTnk\n5W0m3ZtEn4sCid4WcXuMrXNuHDAOoE2bNm7evHnx2nWJMjMzg/9VkCjUFjB3LpxySgZmjoUL/0mj\nRqErCk+fiwKJ0hZWylPaYnGx8zugSaGvG0dfE0kKzsGd0Uv09evnKMQl6cQiyD8BjjSzZmZWDbgS\neD0G2xWJi7ffhtmzoUoVaNBga+hyRMqs3KdWnHM7zOwm4C2gMvC0c+6LclcmEgd5eQVH44ceCpUr\nu7AFieyFmJwjd869CbwZi22JxNOLL8KCBdC4se9yuGFD6IpEyk53dkraysmBu+/288OHQyX9NEiS\n0kdX0ta4cbBsmR8k4uqrQ1cjsvcU5JKW1q8veLb4gw/6C50iyUpBLmnpwQdhzRo44wy45JLQ1YiU\nj4Jc0s4338Djj/v5xx+HUu6xEEkaCnJJO3fe6S909ugBJ50UuhqR8lOQS1r58EP/VMN99oERI0JX\nIxIbCnJJG3l5cMstfv6226BJk12vL5IsFOSSNp55xj8c66CD4I47QlcjEjsKckkLa9fC7bf7+T/9\nCWrVCluPSCwpyCUt3HYb/PQTtGvnL3KKpBIFuaS8zEx/WqV6dRgzRt0NJfUoyCWlbdsGffv6+SFD\n4Mgjw9YjUhEU5JLSRoyAJUv881QGDQpdjUjFUJBLypo7Fx54wJ9KGTvWn1oRSUUKcklJmzfD734H\nubm+7/hZZ4WuSKTiKMglJd15J2RlQYsW/qhcJJUpyCXlzJoFf/6zfzTtpEn+dnyRVKYgl5SyZg30\n6uXn//hHOPHEoOWIxIWCXFJGbq6/2ee77+D00wsGVRZJdQpySRkPPABvvw316sGUKRr1R9KHglxS\nwqxZfgBlM5g8GRo3Dl2RSPwoyCXprVwJV10FzsHdd8Nvfxu6IpH4UpBLUtu0CS6+2F/kbN8ehg0L\nXZFI/CnIJWnl5fmLm599BkccAS++CJUrh65KJP4U5JK0Bg+GV1+FOnVgxgyoWzd0RSJhKMglKT39\nNDz8sD8CnzYNjjoqdEUi4SjIJem8+ipcf72ff+opP1iESDpTkEtSefdd6NrVnx8fNqzgWeMi6UxB\nLkljzhy45BLIyYE//MH3GxcRBbkkifnzoWNH392wZ08YNUpDtonkK1eQm9nlZvaFmeWZWZtYFSVS\n2EcfwTnnwLp1cOml8Le/QSUdgoj8orw/DouATsD7MahFpJj33/d3am7YAJdfDlOnQtWqoasSSSzl\neqyQc+5LANPfuFIBZs6Ezp1hyxZ/48+ECXoQlkhJ4vZjYWZ9gD4ADRs2JDMzM167LlF2dnbwGhJF\nIrbFjBkH8/jjzcnLMzp2XEWvXll88EHF7jMSiZCbm5twbRFKIn4uQkn4tnDO7XICZuFPoRSdLim0\nTibQZnfbyp9at27tQps9e3boEhJGIrVFXp5zQ4c65x+B5dyQIf61eGjbtq1r2bJlfHaWBBLpcxFa\norQFMM+VkKm7PSJ3zrWvoN8hIjvZvNnf6PP88/6OzdGjoU+f0FWJJD6dcZSE8M030KkTLFgANWv6\ngSEuuCB0VSLJobzdDy8zs5XAacDfzeyt2JQl6eSdd6BNGx/iRxwBH3+sEBcpi3IFuXPuFedcY+dc\ndedcQ+dch1gVJqlv+3b/BMMOHeDnn+H88+GTT+DYY0NXJpJcdGpFgli6FLp3h7lz/c09w4b5STf6\niJSdglziyjkYPx5uvRWys6FJEz/G5plnhq5MJHkpyCVuvv7a90qZPdt/ffnlMHYsHHBA2LpEkp3+\nkJUKl5MDjzwCxx3nQ7x+fT8s25QpCnGRWNARuVSof/wD+veHJUv81z16wOOPQ716YesSSSU6IpcK\nsXixH92+Y0cf4s2b+1CfNEkhLhJrCnKJqeXL4dpr/WmUN96AWrX8aZXPP/fdDEUk9nRqRWJi2TIY\nOdL3SMnJ8U8p7NsX7r4bDj44dHUiqU1BLuXyn//40eynTPHjaAJ06wb33uvv0hSRiqcglzLLy/OD\nII8aBW++6V+rUsUPwTZoELRoEbY+kXSjIJc9tnq1H9xh3Dh/KgVg33193/Bbb4VDDw1bn0i6UpDL\nLm3fDu+95wP85Zf91+BD+/rr4YYb1AtFJDQFuRSTlwf//je88AK89BKsXetfr1TJdyns29f3QKlc\nOWydIuIpyAWAbdv8QMczZvgj75UrC5YdfTRcdRX06uWfjSIiiUVBnsZWrYK33oIJE37Np5/6h1jl\nO+wwuPJK3wPl+ONB42uLJC4FeRpZuxYyM/3zTt57D/773/wl9QEf2BdeCBddBKecovAWSRYK8hS1\nYwcsWgRz5hRMixfvvE7NmnDWWdC8+RJuvbW5ep2IJCkFeQrIyfFH159/7m/QmTMH5s3zgxkXVr06\nnHEGnHMOnH02nHQSVK0KmZnfc+ihzcMULyLlpiBPItu2+UGKlyzxR9uff+6nrCx/BF7Ur34Fp57q\nT5Occgq0auXDXERSi4I8gTjnx65cscI/fGrZMvjqq4Jp+fKC2+ALM/O3wx93nD/PfdJJcPLJ/rnf\nIpL6FORxsm2bvzPyxx/99MMPvovf8uUFwb1iRfHTIYVVqgTNmsGRR/rb4I87zk8tWvjz3SKSnhTk\nZeQcbNoE69b5o+d160qeX7u2ILRXr4ZIZM+2v99+/q7JJk2gaVMf2vlTs2Y6NSIixaVkkO/YAVu3\n+mnLlp3/zZ//5JO6/PijPwLeuNH3oS78b0mvZWfD+vUFt6mXReXK0KABNGxY8G+TJn7KD+5DD4X9\n9499e4hIagsS5KtWwbBhPhBjMeXk7BzWJV34K+64va6/Rg0/1uSBB/p/S5o/8EAf1vnTgQf6UyMi\nIrEWJMi//z6L++7LKPLqFUA/YDNwfgnf1Ss6rQW6lLD8RqArsALoSaVK7DQ1aDCQBg0uIi8vi2++\n6Utu7naqV69KpUr+aPmss4Zy7LHtWb9+Aa+/PoDKldlpuuOOEbRtezpffPEh99wzeKc9r18P99wz\nilatWjFr1izuv//+YtWNHTuWo446ijfeeIORI0cWWz5p0iSaNGnClClTGDNmTLHl06ZNo169ekyc\nOJGJEycWW/7mm2+y7777Mnr0aKZOnVpseWZmJgCPPvooM2bM2GnZli1bmDNnDgD33Xcf77777k7L\n69aty/Tp0wG46667+Oijj3Za3rhxY5577jkABgwYwIIFC3Za3rx5c8aNGwdAnz59WJI/gGdUq1at\nGDVqFAA9evRgZeHnAwCnnXYaDz74IACdO3fmp59+2ml5u3btuPvuuwHo2LEjW7Zs2Wn5hRdeyG23\n3QZARkYGRV1xxRX069ePvLw8li5dWmydXr160atXL9auXUuXLsU/ezfeeCNdu3ZlxYoV9OzZs9jy\ngQMHctFFF5GVlUXfvn2LLR86dCjt27dnwYIFDBgwoNjyESNGcPrpp/Phhx8yePDgYstHjaqYz14k\nEqFOnToV+tmrUaMGM2fOBNL7s7d582bOP7947u3us5cvSJBXq+ZHjalUyfe4MIPWraFdO98r44kn\n/GuFl593nr/rcNMmGDq0+PJrr/W3lK9ZA717F9/nwIH+jsWsLP/Qp0hkE3Xq1Pllee/e0L49LFgA\nc+cW//5GjfwpkaVLK7BhRET2gjnn4r7TNm3auHnz5sV9v4VlZmaW+BsyHaktvIyMDCKRSLGjunSl\nz0WBRGkLM5vvnGtT9HWdtRURSXIKchGRJKcgFxFJcgpyEZEkpyAXEUly5QpyM3vEzP5rZgvN7BUz\nq7P77xIRkVgq7xH5O8CxzrnjgSXAXeUvSUREyqJcQe6ce9s5l39D/MdA4/KXJCIiZRHLOzuvBaaU\nttDM+gB9ABo2bPjLbbuhZJxKO5gAAAMWSURBVGdnB68hUagtvEgkQm5urtoiSp+LAoneFrsNcjOb\nBRxUwqIhzrnXousMAXYAk0vbjnNuHDAO/J2doe+SSpQ7tRKB2sKrU6cOkUhEbRGlz0WBRG+L3Qa5\nc679rpabWS/gQqCdC3G/v4hImivXqRUzOw8YBLR1zu1ibBsREako5e218hdgP+AdM1tgZn+NQU0i\nIlIG5Toid84dEatCRERk7+jOThGRJKcgFxFJckEGljCzNcD/4r7jndXDjxsnaovC1BYF1BYFEqUt\nDnPO1S/6YpAgTwRmNq+kkTbSkdqigNqigNqiQKK3hU6tiIgkOQW5iEiSS+cgHxe6gASitiigtiig\ntiiQ0G2RtufIRURSRTofkYuIpAQFuYhIklOQA2Y20MycmdULXUsoGrbPPwTOzLLMbKmZ3Rm6nlDM\nrImZzTazxWb2hZn1D11TaGZW2cw+M7MZoWspSdoHuZk1AX4LLA9dS2BpPWyfmVUGngI6Ai2AbmbW\nImxVwewABjrnWgCnAr9P47bI1x/4MnQRpUn7IAcexz+KN62v+mrYPk4GljrnljnncoAXgUsC1xSE\nc26Vc+7T6PxGfIA1CltVOGbWGLgAGB+6ltKkdZCb2SXAd865/4SuJcFcC8wMXUScNQJWFPp6JWkc\nXvnMrClwAjAnbCVBjcIf7OWFLqQ0sRyzMyHtaqg6YDD+tEpaiNWwfZIezKwWMB0Y4JzbELqeEMzs\nQmC1c26+mWWErqc0KR/kpQ1VZ2bHAc2A/5gZ+FMJn5rZyc65H+JYYtxo2L5d+g5oUujrxtHX0pKZ\nVcWH+GTn3Muh6wnoDOBiMzsf2AeobWbPOed6BK5rJ7ohKMrMvgXaOOcS4QlncRcdtu8x/LB9a0LX\nE29mVgV/kbcdPsA/Aa5yzn0RtLAAzB/ZPAP87JwbELqeRBE9Ir/NOXdh6FqKSutz5LKTtB62L3qh\n9ybgLfzFvanpGOJRZwA9gXOin4UF0SNSSVA6IhcRSXI6IhcRSXIKchGRJKcgFxFJcgpyEZEkpyAX\nEUlyCnIRkSSnIBcRSXL/D85x0YX3vZ1MAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-1p6x0brx4_p",
        "colab_type": "text"
      },
      "source": [
        "In general, the best activation function to use are:\n",
        "1. LU Variants\n",
        "    1. SELU \n",
        "        - if using sequential CNNs\n",
        "    2. ELU\n",
        "        - if using recurrent neural networks\n",
        "    3. Leaky ReLU\n",
        "        - When runtime performance is preferable\n",
        "2. ReLU\n",
        "3. tanh\n",
        "4. Logistic"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cn_dBUTCx9q6",
        "colab_type": "text"
      },
      "source": [
        "Another common activation function is a softmax activation function. This is typically used to ensure the output of the estimated probabilities for class membership at the end of a model are between 0 and 1. It does this by suppressing values below a maximum value so the values sum to 1<sup>1</sup>.\n",
        "\n",
        "**Note**\n",
        "- the below diagram is adapted from<sup>1</sup> which has useful plots of more activation functions.\n",
        "\n",
        "---\n",
        "1.  https://adl1995.github.io/an-overview-of-activation-functions-used-in-neural-networks.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BZEQknrzDrra",
        "colab_type": "code",
        "outputId": "92f61873-ac42-42da-bd78-3ef37e6190fe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 268
        }
      },
      "source": [
        "z = np.linspace(-5, 5, 100)\n",
        "plt.plot(np.exp(z) / np.sum(np.exp(z)))\n",
        "plt.show()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD7CAYAAABkO19ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAekklEQVR4nO3de3gd9X3n8fdXd0u2ZdmWb/JFwjYY\nYwcMwpBAKJeS2myCSRY2UNq4KY0329A0SXlasrvNEtLn2bKbDU0bnjQE0gAN9ybBT0JCuKQNJGBb\nxgTfsbBsy7JlydZdsi5H57t/nJE5ViR8bOuckUaf1/Po0cxvfufMdxjzOaPfzJkxd0dERKIrK+wC\nREQkvRT0IiIRp6AXEYk4Bb2ISMQp6EVEIk5BLyIScSkFvZmtMrPdZlZtZncPsfwqM3vTzGJmdvOg\nZWvNbE/ws3akChcRkdTYqa6jN7Ns4B3geuAgsAm4zd13JPUpByYDdwHr3f3ZoH0qUAVUAg5sBi5x\n9+aR3hARERlaTgp9VgLV7r4XwMyeBNYAJ4Le3fcFy+KDXvsHwIvu3hQsfxFYBTwx3MqmT5/u5eXl\nqW+BiIiwefPmo+5eOtSyVIK+DKhNmj8IXJbiuod6bdn7vaC8vJyqqqoU315ERADMbP9wy0bFyVgz\nW2dmVWZW1djYGHY5IiKRkkrQ1wHzkubnBm2pSOm17v6gu1e6e2Vp6ZB/eYiIyBlKJeg3AYvNrMLM\n8oBbgfUpvv8LwEfMrMTMSoCPBG0iIpIhpwx6d48Bd5II6J3A0+6+3czuNbMbAczsUjM7CNwCfMfM\ntgevbQK+RuLDYhNw78CJWRERyYxTXl6ZaZWVla6TsSIip8fMNrt75VDLRsXJWBERSR8FvYhIxCno\nRURGge+9VsPzWw+n5b0V9CIio8A//8e7vLyzIS3vraAXEQlZ6/E+Gtp7WDxzYlreX0EvIhKy6oYO\nABaVKuhFRCLp3SDodUQvIhJRexraycvJYm5JYVreX0EvIhKy6oYOFpZOJDvL0vL+CnoRkZDtaehg\n0Yz0DNuAgl5EJFTHe/upaznOYgW9iEg0vdvYgTs6ohcRiaoTl1Yq6EVEoqm6oYPsLKN8WlHa1qGg\nFxEJ0Z6GdhZMKyQvJ31xrKAXEQlRdUNHWk/EgoJeRCQ0vbE4+491pXV8HhT0IiKh2X+sk1jcWTxj\nUlrXo6AXEQlJJq64AQW9iEho9jR0YAYL03TXygEKehGRkFQ3dFA2ZQIT8rLTuh4FvYhISNJ9j5sB\nCnoRkRDE+uPsbUz/pZWgoBcRCcW+Y130xOIsmTU57etS0IuIhGBXfRsAS2an99JKUNCLiIRi1+F2\nsrNMY/QiIlG1q76NhaVF5Oek94obUNCLiIRi5+H2jIzPg4JeRCTj2rr7qGs5npHxeVDQi4hk3O76\ndgCWzFLQi4hE0q4TQa+hGxGRSNp1uI3JBTnMLi7IyPpSCnozW2Vmu82s2szuHmJ5vpk9FSzfYGbl\nQXuumT1iZlvNbKeZfXlkyxcRGXt21bezZPZkzCwj6ztl0JtZNvAAsBpYCtxmZksHdbsDaHb3RcD9\nwH1B+y1AvrsvBy4B/uvAh4CIyHgUjzu769s5P0Pj85DaEf1KoNrd97p7L/AksGZQnzXAI8H0s8B1\nlviocqDIzHKACUAv0DYilYuIjEF1Lcfp6IlxXobG5yG1oC8DapPmDwZtQ/Zx9xjQCkwjEfqdwGHg\nAPB1d286y5pFRMasnYczd+uDAek+GbsS6AfmABXAX5nZOYM7mdk6M6sys6rGxsY0lyQiEp6BK27O\nmzm6gr4OmJc0PzdoG7JPMExTDBwD/hD4ubv3uXsD8GugcvAK3P1Bd69098rS0tLT3woRkTFiV30b\nC6YVUpSfk7F1phL0m4DFZlZhZnnArcD6QX3WA2uD6ZuBV9zdSQzXXAtgZkXA5cCukShcRGQs2lXf\nnrEvSg04ZdAHY+53Ai8AO4Gn3X27md1rZjcG3R4GpplZNfAlYOASzAeAiWa2ncQHxr+4+9sjvREi\nImNBV2+MfUc7M3oiFiClvx3c/Xng+UFtX0ma7iZxKeXg13UM1S4iMh7tPNxG3GF5WXFG16tvxoqI\nZMjWg60ALCvL7BG9gl5EJEO21rUxfWIesyZn5tYHAxT0IiIZsq2ulWVlxRm79cEABb2ISAYc7+1n\nT0N7xsfnQUEvIpIRO+sTJ2IvmKOgFxGJpG11iROxy+cq6EVEImlbXStTi/KYk6F70CdT0IuIZMDW\nurZQTsSCgl5EJO26+/rZc6SdZXMye/38AAW9iEia7a5vJxb3UK64AQW9iEjaba0b+Easgl5EJJK2\n1bUypTCXuSUTQlm/gl5EJM22HWpl2ZxwTsSCgl5EJK16Yv3srm8PbdgGFPQiImm141Abff3OB0L4\notQABb2ISBptOdACwMXzS0KrQUEvIpJGW2pbmFNcwKwQvhE7QEEvIpJGWw40syLEo3lQ0IuIpE1D\nezcHm4+zYv6UUOtQ0IuIpMlbwfi8gl5EJKK21LaQm22h3IM+mYJeRCRNthxoZunsyRTkZodah4Je\nRCQNYv1x3j7YGvqJWFDQi4ikxTtHOujq7Q99fB4U9CIiabGlthmAFfN0RC8iEklv7m9h+sQ85k0N\n546VyRT0IiJpsKW2mYvmlYR2x8pkCnoRkRHW0tXL3sbOUTE+Dwp6EZERt3l/Ynw+zBuZJVPQi4iM\nsI01TeRlZ+mIXkQkqjbUNHHhvOLQvyg1QEEvIjKCOntibKtrZWXF1LBLOSGloDezVWa228yqzezu\nIZbnm9lTwfINZlaetOwDZva6mW03s61mFt5NmUVE0mzLgRZicWdlxbSwSznhlEFvZtnAA8BqYClw\nm5ktHdTtDqDZ3RcB9wP3Ba/NAf4V+Ky7XwBcDfSNWPUiIqPMxppjZGcZlywYHSdiIbUj+pVAtbvv\ndfde4ElgzaA+a4BHgulngesscfHoR4C33f23AO5+zN37R6Z0EZHR542aJpbNmczE/JywSzkhlaAv\nA2qT5g8GbUP2cfcY0ApMA84F3MxeMLM3zeyvz75kEZHRqbuvn7dqW0bV+DxAuj9ycoArgUuBLuBl\nM9vs7i8ndzKzdcA6gPnz56e5JBGR9Hj7YCu9sfioGp+H1I7o64B5SfNzg7Yh+wTj8sXAMRJH/79y\n96Pu3gU8D1w8eAXu/qC7V7p7ZWlp6elvhYjIKLCx5hgAl5aPnvF5SC3oNwGLzazCzPKAW4H1g/qs\nB9YG0zcDr7i7Ay8Ay82sMPgA+D1gx8iULiIyumyoaWLJrElMKcwLu5STnDLogzH3O0mE9k7gaXff\nbmb3mtmNQbeHgWlmVg18Cbg7eG0z8A0SHxZvAW+6+09HfjNERMIV64+zeX/zqBufhxTH6N39eRLD\nLsltX0ma7gZuGea1/0riEksRkcjadqiNrt5+Li0ffUGvb8aKiIyAX1cfBeCDC0fXiVhQ0IuIjIhX\n9zSydPZkpk/MD7uU36GgFxE5S129MTbvb+bKxdPDLmVICnoRkbO0saaJvn7nykUKehGRSHptz1Hy\ncrJG5RU3oKAXETlrr1UfpXJByai5//xgCnoRkbPQ0N7Nrvr2UTs+Dwp6EZGz8pvqxG0PPrxo9N6+\nRUEvInIWXt1zlCmFuSydMznsUoaloBcROUPuzmvVjVyxcDrZWRZ2OcNS0IuInKF3Gzs40tYzqsfn\nQUEvInLG/n13I8CovX5+gIJeROQMvbyzgfNmTmLe1MKwS3lfCnoRkTPQeryPTfuauPb8GWGXckoK\nehGRM/DqnkZicee6JQp6EZFIemVnA1MKc1kxf3Q9NnAoCnoRkdPUH3d+ubuBa86bMaovqxygoBcR\nOU1v1TbT3NXHtWNg2AYU9CIip+3lnQ1kZxlXnTt6b3uQTEEvInKaXtnVwKXlJRRPyA27lJQo6EVE\nTkNdy3F21bdz3ZKZYZeSMgW9iMhpeGnHEYAxcf38AAW9iMhpeH7rYc6dOZGFpRPDLiVlCnoRkRQ1\ntvewcV8Tq5fNDruU06KgFxFJ0Qvb63GHG5Yr6EVEIuln2w5zTmkR584cO8M2oKAXEUnJsY4e3tjb\nxA3LZmM2+r8Nm0xBLyKSghd3HKE/7qxePivsUk6bgl5EJAXPb6tnwbRCls4evc+GHY6CXkTkFFq6\nevlN9VFWj8FhG1DQi4ic0i+2HyEWd24Yg8M2oKAXETmlH79Vx4JphSwvKw67lDOioBcReR+HW4/z\n+t5jfHxF2ZgctoEUg97MVpnZbjOrNrO7h1ieb2ZPBcs3mFn5oOXzzazDzO4ambJFRDLjx1sO4Q4f\nX1EWdiln7JRBb2bZwAPAamApcJuZLR3U7Q6g2d0XAfcD9w1a/g3gZ2dfrohI5rg7P9pykEsWlLBg\nWlHY5ZyxVI7oVwLV7r7X3XuBJ4E1g/qsAR4Jpp8FrrPgbxwzuwmoAbaPTMkiIpmx/VAb7xzp4KYx\nfDQPqQV9GVCbNH8waBuyj7vHgFZgmplNBP4G+OrZlyoiklk/2lJHbrbx0TF2b5vB0n0y9h7gfnfv\neL9OZrbOzKrMrKqxsTHNJYmInFqsP85zbx3imvNmUFKUF3Y5ZyUnhT51wLyk+blB21B9DppZDlAM\nHAMuA242s/8DTAHiZtbt7t9KfrG7Pwg8CFBZWelnsiEiIiPpteqjHO3o4RMXj+1hG0gt6DcBi82s\ngkSg3wr84aA+64G1wOvAzcAr7u7Ahwc6mNk9QMfgkBcRGY2eqTrIlMJcrlkydp4kNZxTBr27x8zs\nTuAFIBv4nrtvN7N7gSp3Xw88DDxmZtVAE4kPAxGRMamxvYcXttez9kPl5Odkh13OWUvliB53fx54\nflDbV5Kmu4FbTvEe95xBfSIiGffM5lpicee2lfNO3XkM0DdjRUSSxOPOkxtrWVkxlUUzJoVdzohQ\n0IuIJPn1u0c50NTF7ZfND7uUEaOgFxFJ8viGA5QU5rJq2di8U+VQFPQiIoGG9m5e3HGEmy+ZG4mT\nsAMU9CIigWeqDhKLO7eujM6wDSjoRUQA6I3FefT1fVyxaBoLSyeGXc6IUtCLiADPbz3MkbYe/uzK\nc8IuZcQp6EVk3HN3HnptLwtLi/i9c0vDLmfEKehFZNzbUNPEtro27rjyHLKyxuZTpN6Pgl5Exr2H\nX6uhpDA3EjcwG4qCXkTGtZqjnby08wh/fPkCCnKjc0llMgW9iIxrD7+2l9ysLP7ogwvCLiVtFPQi\nMm4daevm6aqDfOLiMmZMKgi7nLRR0IvIuPWd/9hLf9z586sXhV1KWinoRWRcamjv5gcb9vPxFWXM\nn1YYdjlppaAXkXHpoVdr6OuP87lron00Dwp6ERmHjnX08Njr+1lzURkV04vCLiftFPQiMu5899Ua\numP94+JoHhT0IjLOHGnr5vu/qeHGC+ewaEa0bl42HAW9iIwr//DSO/THnbs+cl7YpWSMgl5Exo09\nR9p5alMtf3T5AuZNjfaVNskU9CIybtz3890U5eXwF9cuDruUjFLQi8i4sGlfEy/tPMJnr17I1KK8\nsMvJKAW9iERePO783U93MmtyAX96RUXY5WScgl5EIu+ZzbX8traFv151HhPyonmHyvejoBeRSGvp\n6uXvf7aLS8tL+PiKaN5v/lQU9CISaf/3hd20dce4d80yzKL39KhUKOhFJLLePtjC4xsPsPaD5Zw/\ne3LY5YRGQS8ikRTrj/M/f7yN6RPz+cL14+tyysEU9CISSd99tYa3D7bylY8uZXJBbtjlhEpBLyKR\ns+dIO/e/+A6rLpjFRz8wO+xyQqegF5FIifXHuevZtynKz+ZrN43fE7DJUgp6M1tlZrvNrNrM7h5i\neb6ZPRUs32Bm5UH79Wa22cy2Br+vHdnyRURO9tBrNfy2toWvrllG6aT8sMsZFU4Z9GaWDTwArAaW\nAreZ2dJB3e4Amt19EXA/cF/QfhT4mLsvB9YCj41U4SIig20/1Mo3fvEOf3DBTD6mIZsTUjmiXwlU\nu/ted+8FngTWDOqzBngkmH4WuM7MzN23uPuhoH07MMHM9BErIiOusyfGXzy+hZKiXP73Jz6gIZsk\nqQR9GVCbNH8waBuyj7vHgFZg2qA+/xl40917zqxUEZHh/e1z29h3rJNv3rpi3N207FRyMrESM7uA\nxHDOR4ZZvg5YBzB//vxMlCQiEfJvmw/ywzfr+MLvL+bycwYfY0oqR/R1wLyk+blB25B9zCwHKAaO\nBfNzgR8Bn3L3d4dagbs/6O6V7l5ZWlp6elsgIuParvo2/va5bVxWMXXc3Wc+VakE/SZgsZlVmFke\ncCuwflCf9SROtgLcDLzi7m5mU4CfAne7+69HqmgREYCmzl4+82gVkwpy+KfbVpCdpXH5oZwy6IMx\n9zuBF4CdwNPuvt3M7jWzG4NuDwPTzKwa+BIwcAnmncAi4Ctm9lbwM2PEt0JExp2+/jif+8GbHGnr\n4Tt/XMmMyQVhlzRqmbuHXcNJKisrvaqqKuwyRGSU+1/PbeOR1/fzjf9yIZ+4eG7Y5YTOzDa7e+VQ\ny/TNWBEZcx5+rYZHXt/PZz5coZBPgYJeRMaU596q42s/2cHqZbO4e/X5YZczJijoRWTMeHVPI3c9\n81tWVkzl/k9epJOvKVLQi8iY8OaBZj772GYWlk7ku5+qpCB3/D379Uwp6EVk1NtyoJm1D29k+qR8\nvv/plRRPGN/3lz9dCnoRGdW2HGjmUw9vpKQojyc+czmzinUZ5elS0IvIqFW1r+lEyD+57nLmTJkQ\ndkljkoJeREall3Yc4faHNlA6KV8hf5YyclMzEZHT8fSmWr78o60smzOZ7/3JpUybqLubnw0FvYiM\nGvG4c/9L7/BPr1Rz1bmlfPv2iynKV0ydLf0XFJFRoaMnxhefeosXdxzhk5Xz+NpNy8jL0ejySFDQ\ni0jo9h3tZN1jVbzb2Mk9H1vK2g+V6wlRI0hBLyKhWv/bQ/z3H24lJ9t45NMruXLx9LBLihwFvYiE\n4nhvP/f+ZDtPbKzlkgUl/ONtKyjTlTVpoaAXkYzbvL+Ju555m5qjnfz51Qv54vXnkput8fh0UdCL\nSMZ09/Xz/36xm4deq2FO8QQe/7PL+NAiDdWkm4JeRDLilV1HuGf9Dg40dXH7ZfP58g3nM1GXTmaE\n/iuLSFrVNnVx70928OKOIywsLeLxz1zGhxbqKD6TFPQikhYtXb1865VqHn19P9lZxt2rl/CnV1To\n2vgQKOhFZER19MR49PV9/PO/v0t7T4xbLpnLF68/l9nFuqImLAp6ERkR7d19PPr6fr776l5auvq4\n5rxS/mb1EpbMmhx2aeOegl5EzsqhluN8/zf7eGLDAdp7Yly7ZAafv24xF82bEnZpElDQi8hpc3fe\n2NvEDzbs52fb6gG4Yfls1n34HJbPLQ65OhlMQS8iKWto7+a5LYd4YtMB9jZ2Mrkgh09/qJw/uaKc\nuSWFYZcnw1DQi8j76uiJ8dKOI/xoSx2v7mkk7rBi/hS+fsuFfPQDs/WQ7jFAQS8iv6O5s5df7m7g\n+a31/GpPI72xOHOKC/hvVy/k4yvmsmjGxLBLlNOgoBcR4nFnZ30br+45yss7j7B5fzNxh1mTC7j9\nsvncsHw2l8wvIStLtw4eixT0IuOQu/NuYycba5p4Y+8xfl19lGOdvQBcMGcyd16ziGuWzODCuVMU\n7hGgoBcZBzp6Ymyra2XLgRbePNDMlgPNHO1IBHvppHyuOreUKxdN58rF05k5uSDkamWkKehFIsTd\naWzvYWd9O7sOt7HjcBtb61qpOdqJe6JPxfQirlpcysqKqVx2zjTKpxXqaU4Rp6AXGYN6Y3Fqm7uo\naexk37FO3m3sYM+RDqobO2jp6jvRb3ZxAcvKirnpojKWlxVz0bwplBTlhVi5hEFBLzIKdfbEqG/r\npr61m7qW49Q1H+dQy3Fqm7uobTrOodbjJ47QAUoKc1k8YxI3LJ/N4hkTWTJrMufPnsSUQoW6pBj0\nZrYK+CaQDTzk7n8/aHk+8ChwCXAM+KS77wuWfRm4A+gHPu/uL4xY9SJjRDzutHX30dzVR1NnL82d\nvTR19nK0s4ej7b0c7eihob2bhvYeGtt6aO+JnfR6M5gxKZ+yKRNYWTGVeVMLWTC1kIrSIiqmFeko\nXd7XKYPezLKBB4DrgYPAJjNb7+47krrdATS7+yIzuxW4D/ikmS0FbgUuAOYAL5nZue7eP9IbIjLS\n3J3e/jjdvXG6+mJ09fbT1dNPV29iuqMnRmdPjI6Bn+4Y7d0x2nv6aO+O0Xq876Sf5CPwZIV52Uyf\nmM+MSfksmTWJqxaXMnNyAbOLC5hVXMCc4gnMKi7Q7X3ljKVyRL8SqHb3vQBm9iSwBkgO+jXAPcH0\ns8C3LHF2Zw3wpLv3ADVmVh283+sjU76Ewd1xBwfi7sQH5h0cJ+6Jdo9D/0nLnX53+uOJ+f64n3h9\nf/y9+VjcT0z3x0/+SSyL09efmO/rjxML2mP9cWL9Tl88Tl/MicXj9PbH6Y3F6etPtPX2v9fWE4vT\nG+unJxanuy9OT6yfnr44x/v66Q5+4sOE81Am5GYzqSAn+MmlpDCP8mlFFE/IpaQwl+LCPKZMyGXq\nxDymFuYxtSiPaRPzKMzTCKqkVyr/wsqA2qT5g8Blw/Vx95iZtQLTgvY3Br227IyrfR+76tu48/Et\np+znwx1WJfc5g4XJzcnrOLk9ub8P3T7c+wcLfIg+jp9oO3n5e4Hs/t4aB0LXgxckLx8I64HXMRDa\nQZ/TCb6wmUFedhZ52Vnk5gz8NvKys8jPySYvJ4u8nCwmFeQyfWIW+TmJ9oLcLApyE78L83IoyM1m\nQm42RfmJ34V5ORTlZzMxP4fC/Bwm5udQlJdNjh5uLaPUqDiUMLN1wDqA+fPnn9F7FORkc97MSSmu\n8Oy6DHcpmp3UZ7h2G7Kdk/on9RnifcxO7nOizd7rOTBtSf0Tv0+uY/CyxPsE726QFUzbSdN24nVZ\nxokv1GRZYn6g73vzRlZW0NeM7KT57CzDgrbsLMjOyiI7K+iXNdBu5GQn3i83Oysxn2XkZGeRk/Xe\n8tysrMTv7KwT/UQktaCvA+Ylzc8N2obqc9DMcoBiEidlU3kt7v4g8CBAZWXlGR0zlk8v4oHbLz6T\nl4qIRFoqf2tuAhabWYWZ5ZE4ubp+UJ/1wNpg+mbgFU+MNawHbjWzfDOrABYDG0emdBERScUpj+iD\nMfc7gRdIXF75PXffbmb3AlXuvh54GHgsONnaROLDgKDf0yRO3MaAz+mKGxGRzLJUTk5mUmVlpVdV\nVYVdhojImGJmm929cqhlukxARCTiFPQiIhGnoBcRiTgFvYhIxCnoRUQibtRddWNmjcD+s3iL6cDR\nESpnrBiP2wzjc7u1zePH6W73AncvHWrBqAv6s2VmVcNdYhRV43GbYXxut7Z5/BjJ7dbQjYhIxCno\nRUQiLopB/2DYBYRgPG4zjM/t1jaPHyO23ZEboxcRkZNF8YheRESSRCbozWyVme02s2ozuzvsetLB\nzOaZ2S/NbIeZbTezvwzap5rZi2a2J/hdEnat6WBm2Wa2xcx+EsxXmNmGYJ8/FdxGOzLMbIqZPWtm\nu8xsp5l9cDzsazP7YvDve5uZPWFmBVHc12b2PTNrMLNtSW1D7l9L+Mdg+982s9N6+EYkgj7pAear\ngaXAbcGDyaMmBvyVuy8FLgc+F2zn3cDL7r4YeDmYj6K/BHYmzd8H3O/ui4BmEg+pj5JvAj939yXA\nhSS2PdL72szKgM8Dle6+jMSt0W8lmvv6+8CqQW3D7d/VJJ7nsZjE0/i+fTorikTQk/QAc3fvBQYe\nYB4p7n7Y3d8MpttJ/I9fRmJbHwm6PQLcFE6F6WNmc4H/BDwUzBtwLYmH0UPEttvMioGrSDzrAXfv\ndfcWxsG+JvGcjAnB0+oKgcNEcF+7+69IPL8j2XD7dw3wqCe8AUwxs9mprisqQT/UA8zT8hDy0cLM\nyoEVwAZgprsfDhbVAzNDKiud/gH4ayAezE8DWtw9FsxHbZ9XAI3AvwTDVQ+ZWRER39fuXgd8HThA\nIuBbgc1Ee18nG27/nlXGRSXoxxUzmwj8G/AFd29LXhY8wjFSl1KZ2UeBBnffHHYtGZQDXAx8291X\nAJ0MGqaJ6L4uIXH0WgHMAYr43eGNcWEk929Ugj6lh5BHgZnlkgj5H7j7D4PmIwN/xgW/G8KqL02u\nAG40s30khuWuJTF+PSX48x6it88PAgfdfUMw/yyJ4I/6vv59oMbdG929D/ghif0f5X2dbLj9e1YZ\nF5WgT+UB5mNeMC79MLDT3b+RtCj54exrgecyXVs6ufuX3X2uu5eT2LevuPvtwC9JPIweIrbd7l4P\n1JrZeUHTdSSevRzpfU1iyOZyMysM/r0PbHdk9/Ugw+3f9cCngqtvLgdak4Z4Ts3dI/ED3AC8A7wL\n/I+w60nTNl5J4k+5t4G3gp8bSIxXvwzsAV4CpoZdaxr/G1wN/CSYPgfYCFQDzwD5Ydc3wtt6EVAV\n7O8fAyXjYV8DXwV2AduAx4D8KO5r4AkS5yH6SPwFd8dw+xcwElcWvgtsJXFVUsrr0jdjRUQiLipD\nNyIiMgwFvYhIxCnoRUQiTkEvIhJxCnoRkYhT0IuIRJyCXkQk4hT0IiIR9/8BIyvfJ0zwagIAAAAA\nSUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O_GcqBFax_Z3",
        "colab_type": "text"
      },
      "source": [
        "## Loss Functions\n",
        "\n",
        "Now we need to measure how far the output of the model is from what we expected. This is were we define a \"*loss function*\" (or objective function) to take the predictions of the network and the true target and compute a score of how distant they are from each other.\n",
        "\n",
        "Defining your loss is important but in general a rule you can use is<sup>1</sup>...\n",
        "\n",
        "- ...binary crossentropy for a two-class classification problems\n",
        "- ...categorical crossentropy for many-class classification problems\n",
        "- ...mean-squared error for regression problems\n",
        "- ...connectionist temporal classification (CTC) for a sequence-learning problems\n",
        "\n",
        "As we are looking at a binary classification problem we will use binary crossentropy. Crossentropy comes from Information Theory and measures the distance between proability distributions; used here to measure the truth from the predictions<sup>1</sup>.\n",
        "\n",
        "The loss function and last-layer activation work together so both should be considered. Heres a table below, again from Chollet, to help<sup>1</sup>:\n",
        "\n",
        "| Problem Type                                            | Last-layer Activation | Loss Function                       |\n",
        "|--------------------------------------------------------------|-----------------------------------|--------------------------------------------|\n",
        "| Binary Classification                                | sigmoid                       | binary_crossentropy            |\n",
        "| Multiclass, single-label Classification  | softmax                        | categorical_crossentropy   |\n",
        "| Multiclass, multilabel Classification     | sigmoid                        | binary_crossentropy             |\n",
        "| Regression to arbitary values                 | None                            | mse                                          |\n",
        "| Regression to values between 0 and 1 | sigmoid                       | mse or binary_crossentropy |\n",
        "\n",
        "**TODO**\n",
        "- mention the different loss function for binary such as hinge detailed in https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/\n",
        "\n",
        "\n",
        "---\n",
        "1. Chollet, F. (2017). Deep learning with python. Manning Publications."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oOQvuYVByBfH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss = 'binary_crossentropy'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rmHLY3clzqKk",
        "colab_type": "text"
      },
      "source": [
        "## Optimizers\n",
        "Once we have scored the output of our model, with our loss function, we want to feedback this back to the connections in the model. This is to adjust the weights so that they will lower the loss score for the current example of data. This adjustment is the job of the optimizer, which implements a Backpropagation algorithm<sup>8</sup>. At the start of a model the network weights are assigned random values, so that it randomly transforms the data. Over multipule examples fed to the system in a training loop, the weights are adjusted slightly in the right direction to decrease the loss. \n",
        "\n",
        "The loss can be thought of as on a slope, with each step change moving futher down the slope. By going against the curvature and updating the weights so you are moving in the opposite direction of the 'gradient', where the slope increases upwards, you intuitively should be lower on the loss curve. The process therefore becomes<sup>8</sup>:\n",
        "\n",
        "1. Draw a batch of training samples x and corresponding targets y\n",
        "2. Run the network on x to obtain predictions y_pred\n",
        "3. Compute the loss of the network on the batch, a measure of the mismatch between y_pred and y\n",
        "4. Compute the gradient of the loss with regard to the network’s parameters (a backward pass)\n",
        "5. Move the parameters a little in the opposite direction from the gradient—for example W -= step * gradient—thus reducing the loss on the batch a bit\n",
        "\n",
        "This process is known as mini-batch stochastic gradient descent (mini-batch SGD). Stochastic refers to each batch of data being drawn at random<sup>8</sup>. Other types of optimisation methods exist, such as those that take into account previous weight updates, rather than just the current value of the gradients, when computing the next weight update<sup>8</sup>. \n",
        "\n",
        "Here are a list of some Optimizers<sup>1<sup>:\n",
        "\n",
        "**Gradient Descent**\n",
        "\n",
        "Takes small regular steps down the slope. It updates the weights by subtracting the cost gradient from the weights, multiplied by the learning rate. \n",
        "\n",
        "**TODO**\n",
        "- improve description, get reference\n",
        "\n",
        "**Momentum Optimization<sup>2</sup>**\n",
        "\n",
        "Momentum Optimization is influenced by the past gradients as at each iteration it subtracts the local gradient from the momentum vector, and adds this momentum vector to the weights. This means the gradient accelerates. To prevent the momentum growing too large, a friction term is set (0 high friction and 1 low). Due to momentum it may oscillate a few times by overshooting and come back before stabalizing at a minimum<sup>1</sup>.\n",
        "\n",
        "*Advantages*\n",
        "- It can escape plateus faster than Gradient Descent.\n",
        "- If Batch Normalization is not used, upper layers will have inputs with different scales which this can help with\n",
        "\n",
        "*Disadvantages*\n",
        "- Add an extra hyperparameter but a 0.9 momentum is a good place to start\n",
        "\n",
        "**Nesterov Accelerated Gradient (Nesterov Momentum optimization)<sup>3</sup>**\n",
        "\n",
        "This is a variant on Momentum Optimization that mesaures the gradient of the cost function ahead in the direction of the momentum rather than the local position.\n",
        "\n",
        "*Advantages*\n",
        "- Nesterov may end up closer to the optimum than Momentum Optimization\n",
        "- Nesterov converges faster than Momentum Optimization as it reduces oscillations\n",
        "\n",
        "**AdaGrad<sup>4</sup>**\n",
        "\n",
        "Gradient Descent goes down the steepest slope and all AdaGrad does is to scale down the gradient vector along the steepest dimensions so that it can correct the direction to point more towards the global optimum. This means the learning rate is decayed faster for steeper dimensions than those with gentler slopes (*adaptive learning rate*).\n",
        "\n",
        "*Advantages*\n",
        "- The learning rate requires less tuning\n",
        "- performs well for simple quadratic problem and with Linear Regression\n",
        "\n",
        "*Disadvantages*\n",
        "- It often stops to early when training neural networks\n",
        "\n",
        "**RMSProp<sup>5</sup>**\n",
        "\n",
        "RMSProp aims to address the problem of stopping to early found in AdaGrad by accumulating the gradients from recent iterations rather than all the previous gradients. This is done using a decay hyperparameter, which 0.9 is a sensible default.\n",
        "\n",
        "*Advantages*\n",
        "- Does not stop to early\n",
        "\n",
        "*Disadvantages*\n",
        "- Adds another hyperparameter\n",
        "\n",
        "**Adaptive Movement Estimation(Adam) Optimization<sup>6</sup>**\n",
        "\n",
        "Combines elements of Momentum optimization and RMSProp by tracking the exponentially decaying average of past gradients as well as tracking the exponentially decaying average of past squared gradients. Again the defaults are pretty good.\n",
        "\n",
        "*Advantages*\n",
        "- Requires less tuning of learning hyperparameter\n",
        "- Defaults are often very good\n",
        "\n",
        "*Disadvantages*\n",
        "- Can lead to solutions that generalize poorly on some datasets<sup>7</sup>\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. Polyak, B. T. (1964). Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics, 4(5), 1-17.\n",
        "3. Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence O (1/k^ 2). In Doklady AN USSR (Vol. 269, pp. 543-547).\n",
        "4. Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul), 2121-2159.\n",
        "5. Tieleman, T., & Hinton, G. (2012). Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), 26-31.\n",
        "6. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.\n",
        "7. Wilson, A. C., Roelofs, R., Stern, M., Srebro, N., & Recht, B. (2017). The marginal value of adaptive gradient methods in machine learning. In Advances in Neural Information Processing Systems (pp. 4148-4158).\n",
        "8.  Chollet, F. (2017). Deep learning with python. Manning Publications."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QJdAazAVz0BG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.optimizers import Adam\n",
        "\n",
        "model.compile(optimizer=Adam(0.001),\n",
        "              loss=loss,\n",
        "              metrics=['accuracy', 'AUC', 'Recall', 'Precision'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6DJNpQKGzcq3",
        "colab_type": "text"
      },
      "source": [
        "Comparison of some optimization methods (animation by [Alec Radford]). The star denotes the global minimum on the error surface.\n",
        "\n",
        "![alt text](https://i.imgur.com/s25RsOr.gif)\n",
        "\n",
        "[Alec Radford]: https://www.reddit.com/r/MachineLearning/comments/2gopfa/visualizing_gradient_optimization_techniques/ckloy0p/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N3NvqsVAz2DJ",
        "colab_type": "text"
      },
      "source": [
        "Lets combine building the model into a function we can call. This will help us later when we want to load in the weights we have saved from training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6iR3N2moJlAG",
        "colab_type": "code",
        "outputId": "098eaced-c633-48f3-fdd1-e4654f1a472a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        }
      },
      "source": [
        "from tensorflow.keras.backend import clear_session\n",
        "\n",
        "# Returns a short sequential model\n",
        "def create_model():\n",
        "  model = Sequential()\n",
        "\n",
        "  model.add(Dense(units=300,\n",
        "                  input_dim=X_train.shape[1],\n",
        "                  activation='relu'))\n",
        "\n",
        "  model.add(Dense(units=100,\n",
        "                  activation='relu'))\n",
        "\n",
        "  model.add(Dense(units=1,\n",
        "                  activation='sigmoid'))\n",
        "\n",
        "  model.compile(optimizer=Adam(0.001),\n",
        "                loss=loss,\n",
        "                metrics=['accuracy', 'AUC', 'Recall', 'Precision'])\n",
        "\n",
        "  return model\n",
        "\n",
        "clear_session()\n",
        "# Create a basic model instance\n",
        "model = create_model()\n",
        "model.summary()"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 300)               389700    \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 100)               30100     \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1)                 101       \n",
            "=================================================================\n",
            "Total params: 419,901\n",
            "Trainable params: 419,901\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nbex8dWQKHdv",
        "colab_type": "text"
      },
      "source": [
        "## Callback Functions\n",
        "\n",
        "It is worth noting that as the models gets better on training data, neural networks eventually start overfitting. This means we should monitor the performance of the model on data that is outside of the training set (validation data). We should also set some parameters to save models when they improve on our chosen metric. We can do this using Callback Functions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sFFD60hKz4Ng",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.callbacks import ModelCheckpoint\n",
        "\n",
        "best_model_filepath = \"Best_MLP_Model.ckpt\"\n",
        "\n",
        "# Save the model under certain conditions\n",
        "callback_checkpoint = ModelCheckpoint(filepath=best_model_filepath,\n",
        "                                      monitor='val_loss',\n",
        "                                      verbose=0,\n",
        "                                      save_weights_only=True,\n",
        "                                      save_best_only=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7FquwUA2z6Cu",
        "colab_type": "text"
      },
      "source": [
        "A regularization method to stop overfitting models is to interrupt training when the performance on the validation set starts dropping. You can evaluate the model on a validation set at regular intervals and save snapshots of the performance. Once a limit on training is reached, you restore the best model snapshot. Although it works well, its best to combine with other regularization methods<sup>1</sup>.\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\"."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t2wyZgKXz8n3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.callbacks import EarlyStopping\n",
        "\n",
        "callback_early_stopping = EarlyStopping(monitor='val_loss',\n",
        "                                        patience=5, \n",
        "                                        verbose=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P-mVOHdNz-Fg",
        "colab_type": "text"
      },
      "source": [
        "**TODO**\n",
        "- use tensorboard more in tutorial"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dV64qeWwz_aO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.python.keras.callbacks import TensorBoard\n",
        "\n",
        "tensorboard_logs_filepath = \"./MLP_logs/\"\n",
        "\n",
        "callback_tensorboard = TensorBoard(log_dir=tensorboard_logs_filepath,\n",
        "                                   histogram_freq=0,\n",
        "                                   write_graph=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NofGrJAW0A2u",
        "colab_type": "text"
      },
      "source": [
        "We can also change the learning rate during the course of training. If a learning rate is too high then training may diverge, and if too low then it will take a very long time to converge. Rather than a constant learning rate, the learning rate can start high and reduce once progress slows<sup>1</sup>.\n",
        "\n",
        "Different methods include:\n",
        "- Predetermined piecewise constant learning rate\n",
        "    - Learning rate changes after x number of epochs\n",
        "- Performance Scheduling\n",
        "    - Measure validation error every n steps and reduce the learning by a factor of λ when error stops dropping\n",
        "- Exponential Scheduling\n",
        "    - Set the learning rate to a function of the iteration number, dropping by a factor of 10 every r steps\n",
        "    - Prefered when compared to the other methods in Senior et al. (2013)<sup>2</sup>.\n",
        "- Power Scheduling\n",
        "    - Similar to Exponential Scheduling but drops slower\n",
        "    \n",
        "We will change the learning rate for the optimizer when the loss does not change. This could be used to try escape local minima or just boost the performance of the model a little bit more at the end of training. We set the learning rate to 1e-3 previously, so we will multiply it by 0.1 to make it 1e-4 once the validation loss doesnt change from an epoch.\n",
        "    \n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. Senior, A., Heigold, G., & Yang, K. (2013, May). An empirical study of learning rates in deep neural networks for speech recognition. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 6724-6728). IEEE."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1n3XpAgn0Dte",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.python.keras.callbacks import ReduceLROnPlateau\n",
        "\n",
        "callback_reduce_lr = ReduceLROnPlateau(monitor='val_loss',\n",
        "                                       factor=0.1,\n",
        "                                       min_lr=1e-4,\n",
        "                                       patience=0,\n",
        "                                       verbose=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XSNi1GPnN-vN",
        "colab_type": "text"
      },
      "source": [
        "We can make another function here to help make it quicker to set up"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D8NPg0axOBOk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def create_callbacks(best_model_filepath, tensorboard_logs_filepath):\n",
        "\n",
        "  callback_checkpoint = ModelCheckpoint(filepath=best_model_filepath,\n",
        "                                        monitor='val_loss',\n",
        "                                        verbose=0,\n",
        "                                        save_weights_only=True,\n",
        "                                        save_best_only=True)\n",
        "  \n",
        "  callback_early_stopping = EarlyStopping(monitor='val_loss',\n",
        "                                        patience=5, \n",
        "                                        verbose=1)\n",
        "  \n",
        "  callback_tensorboard = TensorBoard(log_dir=tensorboard_logs_filepath,\n",
        "                                   histogram_freq=0,\n",
        "                                   write_graph=False)\n",
        "  \n",
        "  callback_reduce_lr = ReduceLROnPlateau(monitor='val_loss',\n",
        "                                       factor=0.1,\n",
        "                                       min_lr=1e-4,\n",
        "                                       patience=0,\n",
        "                                       verbose=1)\n",
        "  \n",
        "  return [callback_checkpoint, callback_early_stopping,\n",
        "          callback_tensorboard, callback_reduce_lr]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gc3tds3v0E7G",
        "colab_type": "text"
      },
      "source": [
        "## Train Model\n",
        "\n",
        "We have unballanced data so lets set a class weight"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UyLZPqor0HLY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.utils import class_weight\n",
        "# calculate the class weights\n",
        "class_weights = class_weight.compute_class_weight('balanced',\n",
        "                                                  np.unique(y_train),\n",
        "                                                  y_train)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uKKC2tUg0H70",
        "colab_type": "text"
      },
      "source": [
        "Now we iterate on the training data by fitting the model, alike to sklearn.\n",
        "\n",
        "We can also specify the validation_data so we can see how the model generalises to data it is not trained on at each epoch\n",
        "\n",
        "**TODO**\n",
        "- Write about how batch size can change performance\n",
        "\n",
        "**NOTE**\n",
        "- It is common to see batchsizes double from 32 (e.g. [32, 64, 128]) depending on data size."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zvxR7pnB0KL2",
        "colab_type": "code",
        "outputId": "99eb8ba2-f5b9-4b68-bdd0-574e921dd255",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 399
        }
      },
      "source": [
        "EPOCHS = 20\n",
        "BATCH_SIZE = 64\n",
        "\n",
        "history = model.fit(X_train, \n",
        "                    y_train,\n",
        "                    batch_size=BATCH_SIZE, \n",
        "                    epochs=EPOCHS,\n",
        "                    validation_split=0.1,\n",
        "                    callbacks= create_callbacks(best_model_filepath, \n",
        "                                                tensorboard_logs_filepath),\n",
        "                    class_weight = class_weights,\n",
        "                    verbose=1)"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 14009 samples, validate on 1557 samples\n",
            "Epoch 1/20\n",
            "   64/14009 [..............................] - ETA: 8:47 - loss: 0.5398 - accuracy: 0.7344 - AUC: 0.7702 - Recall: 1.0000 - Precision: 0.1053WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.104301). Check your callbacks.\n",
            "14009/14009 [==============================] - 4s 320us/sample - loss: 0.0393 - accuracy: 0.9879 - AUC: 0.9844 - Recall: 0.8562 - Precision: 0.8661 - val_loss: 0.0241 - val_accuracy: 0.9917 - val_AUC: 0.9964 - val_Recall: 0.8431 - val_Precision: 0.8958\n",
            "Epoch 2/20\n",
            "13824/14009 [============================>.] - ETA: 0s - loss: 0.0164 - accuracy: 0.9933 - AUC: 0.9990 - Recall: 0.9110 - Precision: 0.9357\n",
            "Epoch 00002: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.\n",
            "14009/14009 [==============================] - 2s 119us/sample - loss: 0.0163 - accuracy: 0.9934 - AUC: 0.9990 - Recall: 0.9118 - Precision: 0.9362 - val_loss: 0.0374 - val_accuracy: 0.9891 - val_AUC: 0.9855 - val_Recall: 0.8824 - val_Precision: 0.8036\n",
            "Epoch 3/20\n",
            "13568/14009 [============================>.] - ETA: 0s - loss: 0.0059 - accuracy: 0.9982 - AUC: 0.9999 - Recall: 0.9764 - Precision: 0.9831\n",
            "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0001.\n",
            "14009/14009 [==============================] - 2s 118us/sample - loss: 0.0060 - accuracy: 0.9981 - AUC: 0.9999 - Recall: 0.9755 - Precision: 0.9803 - val_loss: 0.0346 - val_accuracy: 0.9897 - val_AUC: 0.9860 - val_Recall: 0.8431 - val_Precision: 0.8431\n",
            "Epoch 4/20\n",
            "14009/14009 [==============================] - 2s 121us/sample - loss: 0.0037 - accuracy: 0.9987 - AUC: 1.0000 - Recall: 0.9788 - Precision: 0.9917 - val_loss: 0.0362 - val_accuracy: 0.9872 - val_AUC: 0.9864 - val_Recall: 0.8431 - val_Precision: 0.7818\n",
            "Epoch 5/20\n",
            "14009/14009 [==============================] - 2s 119us/sample - loss: 0.0026 - accuracy: 0.9994 - AUC: 1.0000 - Recall: 0.9935 - Precision: 0.9918 - val_loss: 0.0372 - val_accuracy: 0.9891 - val_AUC: 0.9865 - val_Recall: 0.8431 - val_Precision: 0.8269\n",
            "Epoch 6/20\n",
            "14009/14009 [==============================] - 2s 119us/sample - loss: 0.0019 - accuracy: 0.9997 - AUC: 1.0000 - Recall: 0.9967 - Precision: 0.9967 - val_loss: 0.0402 - val_accuracy: 0.9891 - val_AUC: 0.9861 - val_Recall: 0.8431 - val_Precision: 0.8269\n",
            "Epoch 00006: early stopping\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EnEeAikn0NXY",
        "colab_type": "text"
      },
      "source": [
        "So lets recap. Training a neural network with keras uses the following objects<sup>1</sup>:\n",
        "\n",
        "1. The input data and corresponding targets\n",
        "2. Layers, which are combined into a network (or model) to map the input data to predictions\n",
        "3. The loss function, which defines the feedback signal used for learning by comparing the predictions to the targets\n",
        "4. The optimizer, which determines how learning proceeds by using this loss value to update the network’s weights.\n",
        "5. Choose some metrics to monitor\n",
        "6. Set any callbacks to monitor the model performance\n",
        "7. Iterate on training data by calling the fit() method of the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rORnXmfsIpBf",
        "colab_type": "text"
      },
      "source": [
        "## Evaluate Model\n",
        "\n",
        "Lets have a look at evaluating the models training\n",
        "\n",
        "**TODO**\n",
        "- talk about model performance\n",
        "\n",
        "**NOTES**\n",
        "- code<sup>1</sup>\n",
        "\n",
        "---\n",
        "1. Chollet, F. (2017). Deep learning with python. Manning Publications."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ekGonHDf0awQ",
        "colab_type": "code",
        "outputId": "d9987978-3417-4a8d-998d-f80033ec6f40",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 226
        }
      },
      "source": [
        "list(history.history.keys())"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['loss',\n",
              " 'accuracy',\n",
              " 'AUC',\n",
              " 'Recall',\n",
              " 'Precision',\n",
              " 'val_loss',\n",
              " 'val_accuracy',\n",
              " 'val_AUC',\n",
              " 'val_Recall',\n",
              " 'val_Precision',\n",
              " 'lr']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "80CHe9r20cv2",
        "colab_type": "code",
        "outputId": "88a9166a-6cc5-4597-a3c2-96fb323af201",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "def plot_progress(history_dict):\n",
        "  for key in list(history_dict.keys())[:5]:\n",
        "    plt.clf() # Clears the figure\n",
        "    training_values = history_dict[key] \n",
        "    val_values = history_dict['val_'+key]\n",
        "\n",
        "    epochs = range(1, len(training_values) + 1)\n",
        "\n",
        "    plt.plot(epochs, training_values, 'bo', label='Training '+key)\n",
        "\n",
        "    plt.plot(epochs, val_values, 'b', label='Validation '+key)\n",
        "\n",
        "    if key != 'loss':\n",
        "      plt.ylim([0.,1.1])\n",
        "\n",
        "    plt.title('Training and Validation '+key) \n",
        "    plt.xlabel('Epochs')\n",
        "    plt.ylabel(key)\n",
        "    plt.legend()\n",
        "    plt.show()\n",
        "    \n",
        "plot_progress(history.history)"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxV1b338c+XGQVBASeQweKtgjhG\nrBUHtPpgq6IWxzjU6kO1WrXW3nKt9lFufd3qtWpteWpp1asWRS9eWp460HuFijggARnEoaKiBlCZ\nRBAoBn7PH3sfchKSkAM5OUnO9/16nVfOWXvtlbUT2L+saS9FBGZmZvXVqtAVMDOz5sWBw8zMcuLA\nYWZmOXHgMDOznDhwmJlZThw4zMwsJw4cVlCSWktaK6l3Q+YtJEn9JeVlnnv1siX9VVJpPuoh6WZJ\n923v+XWUe7mkvzV0udZ4HDgsJ+mNO/PaLGl91ucab2B1iYhNEdEpIj5syLxNlaT/kfSzGtK/LWmx\npNa5lBcRJ0fEuAao1zckLapW9r9GxBU7Wra1PA4clpP0xt0pIjoBHwKnZaVtdQOT1Kbxa9mkPQRc\nVEP6RcAfI2JTI9fHLGcOHNagJP1c0uOSHpO0BrhQ0lGSXpH0maSlku6V1DbN30ZSSOqbfv5jevwZ\nSWskvSypX6550+OnSPq7pNWSfi3pRUnfqaXe9anj9yQtlLRK0r1Z57aWdLekFZLeA4bV8SP6L2BP\nSV/POr8b8E3g4fTz6ZLmSPpc0oeSbq7j5z09c03bqkfaRfRm+rN6V9LlaXoX4P8BvbNaj7unv8v/\nyDr/TEkL0p/RFElfzTpWLul6SfPTn/djktrX8XPIrtcQSWXpea9KOjLr2GWSFqV1fk/SeWn6P0ma\nlp6zXNKj9fle1kAiwi+/tusFLAK+US3t58BG4DSSP0w6AkcARwJtgH2BvwNXp/nbAAH0TT//EVgO\nlABtgcdJ/hLPNe/uwBpgeHrseuBL4Du1XEt96vhnoAvQF1iZuXbgamAB0AvoBkxL/mvV+nN7ELgv\n6/NVQFnW5xOAgenP7+D0Gk9Nj/XPLhuYnrmmbdUj/Z3sCyj9HuuBg9Jj3wAW1fC7/I/0/QHA2vS8\ntsCNwNtA2/R4OfAKsGf6vf8OXF7L9V8O/C193x1YDZyf/pwvAlYAuwK7pMf2S/PuBQxI3/8n8JP0\nZ9QBOLrQ/x+K6eUWh+XD9Ij4fxGxOSLWR8TMiJgRERUR8R4wFjiujvMnRERZRHwJjAMO2Y68pwJz\nIuLP6bG7SW7ANapnHf8tIlZHxCLgb1nf6xzg7ogoj4gVwC/qqC8k3VXnZP1FfnGalqnLlIhYkP78\n5gLja6hLTeqsR/o7eS8SU4DngGPqUS7AecCktG5fpmV3IQm2GfdExMfp9/4Ldf/eMk4DFkTEY+nP\n/hHgPeBbmWoDB0rqEBFLI+KNNP1LkgC+V0RsiIgX63kd1gAcOCwfPsr+IGl/SU9J+ljS58Bokr80\na/Nx1vt1QKftyLt3dj0iIkj+Kq5RPetYr+8FfFBHfQGeBz4HTpP0T8ChwGNZdTlK0t8kLZO0muQv\n9Lp+Xhl11kPSqZJmSFop6TPg5HqWmyl7S3kRsZnk59kzK08uv7cay82qd8+I+JykJXIV8LGkv6Q/\nL4AfkbR8ytLusUvqeR3WABw4LB+qTwH9HfA60D8idgF+RtJdkk9LSbpsAJAkqt7kqtuROi4F9sn6\nXOd04TSIPUzS0rgIeDoisltD44EngX0iogvwh3rWpdZ6SOoITAD+DdgjIroCf80qd1vTdpcAfbLK\na0Xy811cj3rVu9xU70y5EfFMRHyDpJtqIcnvibT1cXlE7EUSWMZmj29ZfjlwWGPoTNJX/YWkA4Dv\nNcL3/AtwmKTTlMzsuhbokac6PgFcJ6lnOtD9k3qc8zDJ4PV3yeqmyqrLyojYIOlrJN1EO1qP9kA7\nYBmwSdKpwIlZxz8BukvqXEfZp0s6Pp008GOSMaQZ9axbbf4CDJR0bjoJ4QKScZynJO2V/v52Ihk3\n+wLYDCDpHEmZPwQ+Iwl8npHWSBw4rDH8CLiE5EbzO5JB7LyKiE+Ac4G7SAZbvwK8BvwjD3X8Lcl4\nwXxgJslf9tuq30LgVZIb+lPVDl8J/JuSWWk3kty0d6geEfEZ8ENgIsnA/giSm3bm+OskrZxF6ayp\n3avVdwHJz+e3JMFnGHB6Ot6x3SJiGXA6SZBbkdbx1IhYBbQmCVBL02NfJ2ldQDK2MlPSFyQz1a6K\nZry+p7lR0mo2a9mULKxbAoyIiBcKXR+z5swtDmuxJA2T1DWdvXQzyUycVwtcLbNmz4HDWrIhJFM7\nlwH/CzgzImrrqjKzenJXlZmZ5cQtDjMzy0lRPICue/fu0bdv30JXw8ysWZk1a9byiNhqGntRBI6+\nfftSVlZW6GqYmTUrkmp8CoK7qszMLCcOHGZmlhMHDjMzy4kDh5mZ5cSBw8zMcuLAYWZmOclr4Eif\nFfR2uk/zqBqOt1eyP/XCdIOZvtWOZ/ZAvqG+ZZqZWX7lLXCkTyMdA5wCDADOlzSgWrbLgFUR0Z9k\na8/bqx2/C3gmxzLNzIrap5/CE0/AjTfmp/x8LgAcDCxM929G0nhgOPBGVp7hwC3p+wnAbyQpIkLS\nGcD7JJu35FKmmVlRWbkSnn8epk5NXq+/nqR36gQ//CH0qGsLs+2Qz8DRk6r7H5dTdWP7KnkioiLd\nX7mbpA0kG7ucBNxQU/46ygRA0khgJEDv3nXu5Glm1qysXg0vvJAEiSlTYO5ciICOHWHIELjgAjjh\nBDj8cGiTh7t8U33kyC3A3RGxNtkqOncRMRYYC1BSUpLzI4DHjYOf/hQ+/BB694bbboPS0u2qipnZ\nDlm7Fl58MQkSU6fCrFmweTO0bw9f/zrceisMHQqDB0O7dvmvTz4Dx2Jgn6zPNW1sn8lTnu4L3YVk\ni8gjgRGS7gC6ApvTVsisepS5w8aNg5EjYd265PMHHySfwcHDzPJv/Xp46aXKrqdXX4WKCmjbFo48\nMvmjduhQOOoo6NCh8euXt/040kDwd+BEkpv7TOCCdO/iTJ6rgEERcYWk84CzIuKcauXcAqyNiDvr\nU2ZNSkpKIpeHHPbtmwSL6vr0gUWL6l2MmVm9/OMfMGNGZaB4+WXYuBFat4aSkiRIDB0KRx8NO+/c\nePWSNCsiSqqn563FkY5ZXA1MJtl0/oGIWCBpNFAWEZOA+4FHJC0EVgLnbU+ZDV33D2vZ8r62dDOz\nXHz5JZSVVY5RvPRS0sqQ4NBD4ZprkkAxZAjsskuha7u1otgB0C0OMyukTZvgtdcqxyimT0/GLQAG\nDUoGsocOhWOPhV13LWxdszV6i6M5u+22qmMcADvtlKSbmW3L5s0wb15l19O0aclMKIADDoCLL04C\nxXHHNfxU2cbgwFGDzAC4Z1WZWX1EwBtvVAaK55+HFSuSY/37wznnJIHi+ONhr70KWtUG4cBRi9JS\nBwozq1kEvPNO5RjF3/6WrNaGpEv7tNOS7qfjj4d99qmrpObJgcPMrB7ef78yUEydCkuWJOl77w0n\nn1w586lfv8LWszE4cJiZ1eCjjyq7nqZOrZwws/vulUFi6FDYb79kNlQxceAwMwM+/rhqoFi4MEnf\nbbeky+mGG5JAMWBA8QWK6hw4zKwoLV+ejE1kup/eeitJ79IlmRZ71VVJoBg0CFp556IqHDhsi5Ur\nkwkB69YlK1SHDEkeadCU5pWbba9Vq6o+QXb+/CR9553hmGPgu99NAsWhhyYrtq12XgBoQLJq9aST\nYOZMOPjgZLFSRUXSJB84MAkimWDSp4+b6tY0rF2bzGb65JPkVdP7zNdVq5JzOnRI/h1nxihKSpJn\nQNnWvADQarVpE1x4YfL0zccfT+acf/FF8mC1F19MVrmOGwf33Zfk79mzMogMGZI05fPx6GYrPps3\nJzf4bQWDzPvsRbrZdt01GcTeYw848EA48cTKf7dHHpk8Vda2n/+7F7kIuO46+K//grvvToIGJM33\nzF9kkASX119PgsiLLyZ7ATzxRHKsU6ekSyvTKjnyyCTNDJLnMi1bVr+WwaefJi3d6lq1SlZY77FH\n8urfv/J9JkBk3u++e+M8WryYuauqyN1xB/zkJ3D99fDLX+Z27ocfVrZIpk9P+owjkv7hQw+tbJUc\nfXTLWC1rldat23ZrIPM1s4K6uvbtK2/4NQWA7PfdunmAuhBq66py4Chi48YlXVTnnguPPrrj/zFX\nr04eB51plcyYkYydAOy7b9Vxkv33942gKYmAzz6rfzDIPKCvui5d6g4A2V87d/ZYWVPnwOHAUcVz\nz8EppyQ38mefzU+f78aNMGdOZYvkxRcrH8uw227JzmWZcZLDDy/MhjTFYPNmWLoU3nsvWf38/vvJ\nU54//rhqMPjyy63PlZIuovoEg9139++wpXHgcODYYu7cZPphnz7JWEXXro3zfSOSRVWZIDJ9Orz9\ndnKsXTs44ojKVsnXv550T1j9rFpVNTBkv//gg2SjoAwp6Trca69tdxF17+6pqcXMgcOBA0huIkcd\nldwMXn4ZevUqbH2WLUs2sckEk7Kyyr98DzigskUyZEjyDKBi7dpYvz5pJWSCQfXgkHlkd8auuybd\ng/36Vb4yn/v08awiqx8HDgcOVq5MbsBLliQ36gMPLHSNtrZ+fbKWJNMiefHFypvinntWHSc55JCW\nMw140yYoL689MCxdWjV/hw5Vg0J2YOjXLxlrMNtRBVnHIWkY8CuSbV7/EBG/qHa8PfAwcDiwAjg3\nIhZJGgyMzWQDbomIiek5i4A1wCagoqaLsq1t2ADDh8O778Jf/9o0gwZAx47J4x6OPTb5vHlzss9B\ndvfWhAnJsZ13Tqb+ZlokX/taMuDaFEUkj7ioLTB8+GHVMYZWrZLHcffrB8OGbR0Y9tyzeFtfVnh5\na3FIag38HTgJKAdmAudHxBtZeb4PHBQRV0g6DzgzIs6VtBOwMd1jfC9gLrB3+nkRUBIRy+tbl2Jv\ncWzalMycevLJygV+zdnixVWnAc+dmwSYVq2SVe/ZixN79my8en3xRe2B4f33t56J1KNH7S2G3r29\nmtkKrxAtjsHAwoh4L63AeGA48EZWnuHALen7CcBvJCkisteDdgBafn9ankTAD3+YBI277mr+QQOS\nYHDOOZXXsmYNvPJKZTB58EH4zW+SY336VB0nGTBg+6cBf/ll8qjt2gJDZsZYxs47VwaCE07YumvJ\niyStucpn4OgJfJT1uRw4srY8aWtiNdANWC7pSOABoA9wUURk1pMG8FdJAfwuIsZSA0kjgZEAvXv3\nbpgraobuvBN+/etkgd8Pf1jo2uRH587Jc7ZOOin5XFGRtEIyLZLnnkvWrEAyg+zrX69slRxxRNI9\nBkmQ/eST2gPDRx8lrbeMNm2SlkG/fkk3YPXWQ/fu7k6ylqnJDi1GxAxgoKQDgIckPRMRG4AhEbFY\n0u7Af0t6KyKm1XD+WNJxkpKSkqJssTz6KPzzPyfdVP/+74WuTeNp0yZZF3L44XDttUlAeP/9quMk\nTz+d5G3bNuneWrcuyZNZsJix555JIDj66K0DQ8+eLWdw3iwX+fxnvxjI3m23V5pWU55ySW2ALiSD\n5FtExJuS1gIHAmURsThN/1TSRJIusa0CR7F77jn4zneSDWgeeqi4V2lLyY1+333h4ouTtJUrK6cB\nz5yZDERnBqEzgaFv38rWiJlVymfgmAnsJ6kfSYA4D7igWp5JwCXAy8AIYEpERHrOR2n3VR9gf2CR\npJ2BVhGxJn1/MjA6j9fQLM2dC2eeCV/9Kkyc6Dn7NdltNzj11ORlZrnJW+BIb/pXA5NJpuM+EBEL\nJI0maTlMAu4HHpG0EFhJElwAhgCjJH0JbAa+HxHLJe0LTFTScdwGeDQins3XNTRHH3yQPEqkSxd4\n5pnGWxVuZsXDCwBbkOawwM/Mmg9v5NTCZS/wmzzZQcPM8seBowXI7OA3fTqMH58MiJuZ5UsRz7Vp\nGaov8Dv33ELXyMxaOgeOZi6zwO+HP2y5C/zMrGlx4GjGMgv8zjknCSBmZo3BgaOZyizwO+44ePjh\n4l7gZ2aNy7ebZmjuXDjrLPinf4I//ckL/MyscTlwNDMffgjf/GbyYD8v8DOzQvB03GZk5crkeUpf\nfJHsFb7PPts+x8ysoTlwNBMbNsAZZ1Qu8Bs0qNA1MrNi5cDRDGzaBBddlLQyvMDPzArNYxxNXESy\nCdOECfDLX3qBn5kVngNHE/fLX8K99yaL+66/vtC1MTNz4GjSHn0UfvxjL/Azs6bFgaOJmjKlcoFf\nse/gZ2ZNi29HTdC8eckOfpkFfh06FLpGZmaVHDiamA8/THbw8wI/M2uq8ho4JA2T9LakhZJG1XC8\nvaTH0+MzJPVN0wdLmpO+5ko6s75lNmerViUL/NauTYKGF/iZWVOUt8AhqTUwBjgFGACcL2lAtWyX\nAasioj9wN3B7mv46UBIRhwDDgN9JalPPMpul7B38/vQnL/Azs6Yrny2OwcDCiHgvIjYC44Hh1fIM\nBx5K308ATpSkiFgXERVpegcgszF6fcpsdjZvrlzg99BDMHRooWtkZla7fAaOnsBHWZ/L07Qa86SB\nYjXQDUDSkZIWAPOBK9Lj9SmT9PyRksoklS1btqwBLic/Mjv4ZRb4nXdeoWtkZla3Jjs4HhEzImIg\ncATwL5JymlsUEWMjoiQiSnr06JGfSjaAzAK/667zAj8zax7yGTgWA9nDu73StBrzSGoDdAFWZGeI\niDeBtcCB9Syz2XjsscoFfr/8ZaFrY2ZWP/kMHDOB/ST1k9QOOA+YVC3PJOCS9P0IYEpERHpOGwBJ\nfYD9gUX1LLNZmDIFLrkEjj3WC/zMrHnJ29NxI6JC0tXAZKA18EBELJA0GiiLiEnA/cAjkhYCK0kC\nAcAQYJSkL4HNwPcjYjlATWXm6xryxQv8zKw5U0RsO1czV1JSEmVlZYWuBpAs8DvqKJDg5Ze9VsPM\nmi5JsyKipHq69+NoRKtWJavC166F6dMdNMyseXLgaCSZBX4LF8Kzz3qBn5k1Xw4cjSB7gd9jj3mB\nn5k1b57Lk2fZO/jdeacX+JlZ8+fAkWd33QW/+pUX+JlZy+HAkUePPQY33ABnn50s8JMKXSMzsx3n\nwJEn2Qv8Hn7YC/zMrOXw7SwPvMDPzFoyB44GVn0Hv113LXSNzMwalqfjNiAv8DOzYuDA0UA2bIAz\nzoB33oHJk73Az8xaLgeOBrB5M1x8MUyb5gV+ZtbyeYxjB2UW+P3nf3qBn5kVBweOHZRZ4HfttV7g\nZ2bFwYFjB4wfX7nA7667vMDPzIqDA8d2mjo1GdfwAj8zKza+3W2H+fOTGVT77ecFfmZWfPIaOCQN\nk/S2pIWSRtVwvL2kx9PjMyT1TdNPkjRL0vz06wlZ5/wtLXNO+to9n9dQ3UcfVS7we/ZZL/Azs+KT\nt+m4kloDY4CTgHJgpqRJEfFGVrbLgFUR0V/SecDtwLnAcuC0iFgi6UCSPcZ7Zp1XGhGNvhdsZoHf\nmjVe4GdmxSufLY7BwMKIeC8iNgLjgeHV8gwHHkrfTwBOlKSIeC0ilqTpC4COktrnsa7blFng9/e/\nJ91TXuBnZsUqn4GjJ/BR1udyqrYaquSJiApgNdCtWp5vA7Mj4h9ZaQ+m3VQ3SzXPZZI0UlKZpLJl\ny5btyHVUWeD38MNe4Gdmxa1JD45LGkjSffW9rOTSiBgEHJO+Lqrp3IgYGxElEVHSo0ePHarHj37k\nBX5mZhn5DByLgexRgF5pWo15JLUBugAr0s+9gInAxRHxbuaEiFicfl0DPErSJZY3d90F99zjBX5m\nZhn5DBwzgf0k9ZPUDjgPmFQtzyTgkvT9CGBKRISkrsBTwKiIeDGTWVIbSd3T922BU4HX83UB48cn\nrQ0v8DMzq5S3wJGOWVxNMiPqTeCJiFggabSk09Ns9wPdJC0ErgcyU3avBvoDP6s27bY9MFnSPGAO\nSYvl9/mo/8aN8LOfeYGfmVl1iohC1yHvSkpKoqws99m7S5ZAx45eq2FmxUnSrIgoqZ7ux6rXYe+9\nC10DM7Omxx0wZmaWEwcOMzPLiQOHmZnlxIHDzMxy4sBhZmY5ceAwM7OcOHCYmVlO6hU4JF0raRcl\n7pc0W9LJ+a6cmZk1PfVtcXw3Ij4HTgZ2JXki7S/yViszM2uy6hs4Mo/3+ybwSEQsyEozM7MiUt/A\nMUvSX0kCx2RJnYHN+auWmZk1VfV9VtVlwCHAexGxTtJuwKX5q5aZmTVV9W1xHAW8HRGfSboQuIlk\nm1czMysy9Q0cvwXWSToY+BHwLvBw3mplZmZNVn0DR0UkG3cMB34TEWOAzvmrlpmZNVX1DRxrJP0L\nyTTcpyS1Atpu6yRJwyS9LWmhpFE1HG8v6fH0+AxJfdP0kyTNkjQ//XpC1jmHp+kLJd0reUNXM7PG\nVN/AcS7wD5L1HB8DvYB/r+sESa2BMcApwADgfEkDqmW7DFgVEf2Bu4Hb0/TlwGkRMYhkT/JHss75\nLfC/gf3S17B6XoOZmTWAegWONFiMA7pIOhXYEBHbGuMYDCyMiPciYiMwnqSrK9tw4KH0/QTgREmK\niNciYkmavgDomLZO9gJ2iYhX0q6zh4Ez6nMNZmbWMOr7yJFzgFeBs4FzgBmSRmzjtJ7AR1mfy9O0\nGvNERAXJTK1u1fJ8G5gdEf9I85dvo0wzM8uj+q7j+ClwRER8CiCpB/A/JK2EvJE0kKT7KufnYkka\nCYwE6N27dwPXzMyseNV3jKNVJmikVtTj3MXAPlmfe6VpNeaR1AbokpaNpF7ARODiiHg3K3+vbZQJ\nQESMjYiSiCjp0aPHNqpqZmb1Vd/A8aykyZK+I+k7wFPA09s4Zyawn6R+ktoB5wGTquWZRDL4DTAC\nmBIRIalr+j1GRcSLmcwRsRT4XNLX0tlUFwN/ruc1mJlZA6hXV1VE/FjSt4Gj06SxETFxG+dUSLoa\nmAy0Bh6IiAWSRgNlETEJuB94RNJCYCVJcAG4GugP/EzSz9K0k9NWz/eB/wA6As+kLzMzayRKJie1\nbCUlJVFWVlboapiZNSuSZkVESfX0OlscktYANUUWARERuzRQ/czMrJmoM3BEhB8rYmZmVXjPcTMz\ny4kDh5mZ5cSBw8zMcuLAYWZmOXHgMDOznDhwmJlZThw4zMwsJw4cZmaWEwcOMzPLiQOHbTFuHPTt\nC61aJV/HjSt0jcysKarvRk7Wwo0bByNHwrp1yecPPkg+A5SWFq5eZtb0uMVhAPz0p5VBI2PduiTd\nzCybA4cB8OGHuaWbWfFy4DAAatuW3du1m1l1DhwGwG23wU47VU3baack3cwsW14Dh6Rhkt6WtFDS\nqBqOt5f0eHp8hqS+aXo3SVMlrZX0m2rn/C0tc0762j2f11AsSkth7Fjo0wek5OvYsR4YN7Ot5W1W\nlaTWwBjgJKAcmClpUkS8kZXtMmBVRPSXdB5wO3AusAG4GTgwfVVXGhHeC7aBlZY6UJjZtuWzxTEY\nWBgR70XERmA8MLxanuHAQ+n7CcCJkhQRX0TEdJIAYmZmTUg+A0dP4KOsz+VpWo15IqICWA10q0fZ\nD6bdVDdLUk0ZJI2UVCapbNmyZbnX3szMatQcB8dLI2IQcEz6uqimTBExNiJKIqKkR48ejVpBM7OW\nLJ+BYzGwT9bnXmlajXkktQG6ACvqKjQiFqdf1wCPknSJmZlZI8ln4JgJ7Cepn6R2wHnApGp5JgGX\npO9HAFMiImorUFIbSd3T922BU4HXG7zmZmZWq7zNqoqICklXA5OB1sADEbFA0migLCImAfcDj0ha\nCKwkCS4ASFoE7AK0k3QGcDLwATA5DRqtgf8Bfp+vazAzs62pjj/wW4ySkpIoK/PsXTOzXEiaFREl\n1dOb4+C4mZkVkAOHmZnlxIHDzMxy4sBhZmY5ceAwM7OcOHCYmVlOHDjMzCwnDhxmZpYTBw4zM8uJ\nA4eZmeXEgcPMzHLiwGFmZjlx4DAzs5w4cJiZWU4cOMzMLCcOHGZmlhMHDjMzy0leA4ekYZLelrRQ\n0qgajreX9Hh6fIakvml6N0lTJa2V9Jtq5xwuaX56zr2SlM9rMDOzqvIWOCS1BsYApwADgPMlDaiW\n7TJgVUT0B+4Gbk/TNwA3AzfUUPRvgf8N7Je+hjV87c3MrDb5bHEMBhZGxHsRsREYDwyvlmc48FD6\nfgJwoiRFxBcRMZ0kgGwhaS9gl4h4JZLN0h8GzsjjNZiZWTX5DBw9gY+yPpenaTXmiYgKYDXQbRtl\nlm+jTAAkjZRUJqls2bJlOVbdzMxq02IHxyNibESURERJjx49Cl0dM7MWI5+BYzGwT9bnXmlajXkk\ntQG6ACu2UWavbZRpZmZ5lM/AMRPYT1I/Se2A84BJ1fJMAi5J348ApqRjFzWKiKXA55K+ls6muhj4\nc8NX3czMatMmXwVHRIWkq4HJQGvggYhYIGk0UBYRk4D7gUckLQRWkgQXACQtAnYB2kk6Azg5It4A\nvg/8B9AReCZ9mZlZI1Edf+C3GCUlJVFWVlboapiZNSuSZkVESfX0Fjs4bmZm+eHAYWZmOXHgMDOz\nnDhwmJlZThw4zMwsJw4cZmaWEwcOMzPLiQOHmZnlJG8rx5u6L7/8kvLycjZs2LDtzFZwHTp0oFev\nXrRt27bQVTErekUbOMrLy+ncuTN9+/bFmwg2bRHBihUrKC8vp1+/foWujlnRK9quqg0bNtCtWzcH\njWZAEt26dXPr0KyJKNrAAThoNCP+XZk1HUUdOMzGjYO+faFVq+TruHGFrpFZ0+fAUU8NfYNZsWIF\nhxxyCIcccgh77rknPXv23PJ548aN9Srj0ksv5e23364zz5gxYxjXQHfDIUOGMGfOnAYpqykYNw5G\njoQPPoCI5OvIkQ4eZttStIPjucjcYNatSz5nbjAApaXbV2a3bt223IRvueUWOnXqxA033FAlT0QQ\nEbRqVXN8f/DBB7f5fa666qrtq2AR+OlPK3+nGevWJenb+3s1KwZucdRDXTeYhrZw4UIGDBhAaWkp\nAwcOZOnSpYwcOZKSkhIGDhzI6NGjt+TNtAAqKiro2rUro0aN4uCDD+aoo47i008/BeCmm27innvu\n2ZJ/1KhRDB48mK9+9au89IBAOSAAAA++SURBVNJLAHzxxRd8+9vfZsCAAYwYMYKSkpJttiz++Mc/\nMmjQIA488EBuvPFGACoqKrjooou2pN97770A3H333QwYMICDDjqICy+8sMF/Ztvrww9zSzezhFsc\n9dDYN5i33nqLhx9+mJKSZP+UX/ziF+y2225UVFQwdOhQRowYwYABA6qcs3r1ao477jh+8YtfcP31\n1/PAAw8watSorcqOCF599VUmTZrE6NGjefbZZ/n1r3/NnnvuyZNPPsncuXM57LDD6qxfeXk5N910\nE2VlZXTp0oVvfOMb/OUvf6FHjx4sX76c+fPnA/DZZ58BcMcdd/DBBx/Qrl27LWlNQe/eSeuxpnQz\nq11eWxyShkl6W9JCSVvdxSS1l/R4enyGpL5Zx/4lTX9b0v/KSl8kab6kOZIaZVu/2m4k+brBfOUr\nX9kSNAAee+wxDjvsMA477DDefPNN3njjja3O6dixI6eccgoAhx9+OIsWLaqx7LPOOmurPNOnT+e8\n85Jdew8++GAGDhxYZ/1mzJjBCSecQPfu3Wnbti0XXHAB06ZNo3///rz99ttcc801TJ48mS5dugAw\ncOBALrzwQsaNG9ekFvDddhvstFPVtJ12StLNrHZ5CxySWgNjgFOAAcD5kgZUy3YZsCoi+gN3A7en\n5w4g2X98IDAM+L9peRlDI+KQmrY0zIfGvsHsvPPOW96/8847/OpXv2LKlCnMmzePYcOG1bieoV27\ndlvet27dmoqKihrLbt++/TbzbK9u3boxb948jjnmGMaMGcP3vvc9ACZPnswVV1zBzJkzGTx4MJs2\nbWrQ77u9Skth7Fjo0wek5OvYsR7fMNuWfLY4BgMLI+K9iNgIjAeGV8szHHgofT8BOFHJhP3hwPiI\n+EdEvA8sTMsriELeYD7//HM6d+7MLrvswtKlS5k8eXKDf4+jjz6aJ554AoD58+fX2KLJduSRRzJ1\n6lRWrFhBRUUF48eP57jjjmPZsmVEBGeffTajR49m9uzZbNq0ifLyck444QTuuOMOli9fzrrqA0YF\nVFoKixbB5s3JVwcNs23L5xhHT+CjrM/lwJG15YmICkmrgW5p+ivVzu2Zvg/gr5IC+F1EjK3pm0sa\nCYwE6N0AfUqlpYW5qRx22GEMGDCA/fffnz59+nD00Uc3+Pf4wQ9+wMUXX8yAAQO2vDLdTDXp1asX\n//qv/8rxxx9PRHDaaafxrW99i9mzZ3PZZZcREUji9ttvp6KiggsuuIA1a9awefNmbrjhBjp37tzg\n12BmjUcRkZ+CpRHAsIi4PP18EXBkRFydlef1NE95+vldkuByC/BKRPwxTb8feCYiJkjqGRGLJe0O\n/Dfwg4iYVlddSkpKoqys6nDIm2++yQEHHNBAV9u8VVRUUFFRQYcOHXjnnXc4+eSTeeedd2jTpmnN\nnfDvzKxxSZpV05BAPu8Mi4F9sj73StNqylMuqQ3QBVhR17kRkfn6qaSJJF1YdQYOq9vatWs58cQT\nqaioICL43e9+1+SChpk1Hfm8O8wE9pPUj+Smfx5wQbU8k4BLgJeBEcCUiAhJk4BHJd0F7A3sB7wq\naWegVUSsSd+fDIzGdkjXrl2ZNWtWoathZs1E3gbHI6ICuBqYDLwJPBERCySNlnR6mu1+oJukhcD1\nwKj03AXAE8AbwLPAVRGxCdgDmC5pLvAq8FREPJuvazBrifx8LttRee2PiIingaerpf0s6/0G4Oxa\nzr0NuK1a2nvAwQ1fU7PikI/H51jx8SNHzIpIYz4+x1ouBw6zIuLnc1lDcOAokKFDh261mO+ee+7h\nyiuvrPO8Tp06AbBkyRJGjBhRY57jjz+e6tOPq7vnnnuqLMT75je/2SDPkbrlllu48847d7gcy4/G\nfnyOtUwOHAVy/vnnM378+Cpp48eP5/zzz6/X+XvvvTcTJkzY7u9fPXA8/fTTdO3adbvLs+ahGJ/P\n5ckADc+T9YHrroOG3p/okEMgfZp5jUaMGMFNN93Exo0badeuHYsWLWLJkiUcc8wxrF27luHDh7Nq\n1Sq+/PJLfv7znzN8eNWntSxatIhTTz2V119/nfXr13PppZcyd+5c9t9/f9avX78l35VXXsnMmTNZ\nv349I0aM4NZbb+Xee+9lyZIlDB06lO7duzN16lT69u1LWVkZ3bt356677uKBBx4A4PLLL+e6665j\n0aJFnHLKKQwZMoSXXnqJnj178uc//5mOHTvWeo1z5szhiiuuYN26dXzlK1/hgQceYNddd+Xee+/l\nvvvuo02bNgwYMIDx48fz/PPPc+211wLJNrHTpk3zCvM8yAyA//SnSfdU795J0GipA+OeDJAfbnEU\nyG677cbgwYN55plngKS1cc455yCJDh06MHHiRGbPns3UqVP50Y9+RF0r/H/729+y00478eabb3Lr\nrbdWWZNx2223UVZWxrx583j++eeZN28e11xzDXvvvTdTp05l6tSpVcqaNWsWDz74IDNmzOCVV17h\n97//Pa+99hqQPHDxqquuYsGCBXTt2pUnn3yyzmu8+OKLuf3225k3bx6DBg3i1ltvBZLHxL/22mvM\nmzeP++67D4A777yTMWPGMGfOHF544YU6A5LtmGJ6PpcnA+SHWxzU3TLIp0x31fDhwxk/fjz3338/\nkOyZceONNzJt2jRatWrF4sWL+eSTT9hzzz1rLGfatGlcc801ABx00EEcdNBBW4498cQTjB07loqK\nCpYuXcobb7xR5Xh106dP58wzz9zyhN6zzjqLF154gdNPP51+/fpxyCGHAHU/uh2S/UE+++wzjjvu\nOAAuueQSzj777C11LC0t5YwzzuCMM84AkgctXn/99ZSWlnLWWWfRq1ev+vwIzerkyQD54RZHAQ0f\nPpznnnuO2bNns27dOg4//HAAxo0bx7Jly5g1axZz5sxhjz32qPFR6tvy/vvvc+edd/Lcc88xb948\nvvWtb21XORmZR7LDjj2W/amnnuKqq65i9uzZHHHEEVRUVDBq1Cj+8Ic/sH79eo4++mjeeuut7a6n\nWUaxTgbI97iOA0cBderUiaFDh/Ld7363yqD46tWr2X333Wnbti1Tp07lg5q2qcty7LHH8uijjwLw\n+uuvM2/ePCB5JPvOO+9Mly5d+OSTT7Z0iwF07tyZNWvWbFXWMcccw5/+9CfWrVvHF198wcSJEznm\nmGNyvrYuXbqw66678sILLwDwyCOPcNxxx7F582Y++ugjhg4dyu23387q1atZu3Yt7777LoMGDeIn\nP/kJRxxxhAOHNYhinQwwcmQynhNROa7TkMHDXVUFdv7553PmmWdWmWFVWlrKaaedxqBBgygpKWH/\n/fevs4wrr7ySSy+9lAMOOIADDjhgS8vl4IMP5tBDD2X//fdnn332qfJI9pEjRzJs2LAtYx0Zhx12\nGN/5zncYPDjZ/uTyyy/n0EMPrbNbqjYPPfTQlsHxfffdlwcffJBNmzZx4YUXsnr1aiKCa665hq5d\nu3LzzTczdepUWrVqxcCBA7fsZmi2I4ptMgDUPa7TUNedt8eqNyV+rHrL4N+Z2ba1apW0NKqTkgkR\nuajtseruqjIza0EaY1zHgcPMrAVpjHGdog4cxdBN11L4d2VWP6WlMHYs9OmTdE/16ZN8bshxnaId\nHO/QoQMrVqygW7duSCp0dawOEcGKFSvo0KFDoati1iyUluZ3AkDRBo5evXpRXl7OsmXLCl0Vq4cO\nHTp4UaBZE1G0gaNt27b069ev0NUwM2t2inqMw8zMcufAYWZmOXHgMDOznBTFynFJy4C6H/hUu+7A\n8gasTnPgay4OxXbNxXa9sOPX3CcielRPLIrAsSMkldW05L4l8zUXh2K75mK7XsjfNburyszMcuLA\nYWZmOXHg2Laxha5AAfiai0OxXXOxXS/k6Zo9xmFmZjlxi8PMzHLiwGFmZjlx4KiFpAckfSrp9ULX\npTFI2kfSVElvSFog6dpC1ynfJHWQ9Kqkuek131roOjUWSa0lvSbpL4WuS2OQtEjSfElzJJVt+4zm\nT1JXSRMkvSXpTUlHNVjZHuOomaRjgbXAwxFxYKHrk2+S9gL2iojZkjoDs4AzIuKNAlctb5Q8T3/n\niFgrqS0wHbg2Il4pcNXyTtL1QAmwS0ScWuj65JukRUBJRBTNAkBJDwEvRMQfJLUDdoqIzxqibLc4\nahER04CVha5HY4mIpRExO32/BngT6FnYWuVXJNamH9umrxb/l5SkXsC3gD8Uui6WH5K6AMcC9wNE\nxMaGChrgwGE1kNQXOBSYUdia5F/aZTMH+BT474ho8dcM3AP8M7C50BVpRAH8VdIsSSMLXZlG0A9Y\nBjyYdkn+QdLODVW4A4dVIakT8CRwXUR8Xuj65FtEbIqIQ4BewGBJLbpbUtKpwKcRMavQdWlkQyLi\nMOAU4Kq0K7olawMcBvw2Ig4FvgBGNVThDhy2RdrP/yQwLiL+q9D1aUxpM34qMKzQdcmzo4HT0z7/\n8cAJkv5Y2CrlX0QsTr9+CkwEBhe2RnlXDpRntaAnkASSBuHAYcCWgeL7gTcj4q5C16cxSOohqWv6\nviNwEvBWYWuVXxHxLxHRKyL6AucBUyLiwgJXK68k7ZxO+CDtrjkZaNGzJSPiY+AjSV9Nk04EGmyi\nS9FuHbstkh4Djge6SyoH/k9E3F/YWuXV0cBFwPy0zx/gxoh4uoB1yre9gIcktSb5I+qJiCiK6alF\nZg9gYvK3EW2ARyPi2cJWqVH8ABiXzqh6D7i0oQr2dFwzM8uJu6rMzCwnDhxmZpYTBw4zM8uJA4eZ\nmeXEgcPMzHLiwGG2nSRtSp+2mnk12MpcSX2L5cnM1vx4HYfZ9lufPq7ErKi4xWHWwNK9H+5I9394\nVVL/NL2vpCmS5kl6TlLvNH0PSRPTfUHmSvp6WlRrSb9P9wr5a7q6HUnXpPumzJM0vkCXaUXMgcNs\n+3Ws1lV1btax1RExCPgNydNoAX4NPBQRBwHjgHvT9HuB5yPiYJLnCS1I0/cDxkTEQOAz4Ntp+ijg\n0LScK/J1cWa18cpxs+0kaW1EdKohfRFwQkS8lz448uOI6CZpOclmWV+m6UsjorukZUCviPhHVhl9\nSR7zvl/6+SdA24j4uaRnSTYZ+xPwp6w9RcwahVscZvkRtbzPxT+y3m+ickzyW8AYktbJTEkeq7RG\n5cBhlh/nZn19OX3/EskTaQFKgRfS988BV8KWjaW61FaopFbAPhExFfgJ0AXYqtVjlk/+S8Vs+3XM\nepIwwLMRkZmSu6ukeSSthvPTtB+Q7Mj2Y5Ld2TJPK70WGCvpMpKWxZXA0lq+Z2vgj2lwEXBvQ24J\nalYfHuMwa2DpGEdJRCwvdF3M8sFdVWZmlhO3OMzMLCducZiZWU4cOMzMLCcOHGZmlhMHDjMzy4kD\nh5mZ5eT/A/jULUFjciSaAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZgU5bn+8e/NooAIKGCMgEAiUdkG\nhhE04oJbcI8rIsaIQRJPRGMSc/glRo3GnBijURPjkRi3iCzH3cQlgiTEY6IssiiocHQQEBUQEUSD\n6PP7o2rGZpilgekZZ+r+XFdfXfXWW1VP9fTU0/VW1VuKCMzMLLua1HcAZmZWv5wIzMwyzonAzCzj\nnAjMzDLOicDMLOOcCMzMMs6JwDYjqamk9ZL2rM269UnSXpIKcp10xWVL+qukEYWIQ9JPJf33ts5v\nVhUnggYu3RGXvT6V9GHOeKU7pOpExCcR0Toi3qjNup9XkqZIuqyS8lMkLZfUdGuWFxFHRcT4Wojr\nCEmlFZZ9VUR8Z3uXbVaRE0EDl+6IW0dEa+AN4Picsi12SJKa1X2Un2t3Ad+opPwbwD0R8Ukdx5M5\n/k7WPyeCRk7SzyVNkjRB0jrgLEkHSPqXpPckrZB0k6Tmaf1mkkJSt3T8nnT645LWSfqnpO5bWzed\nfrSkVyWtlfRbSf8r6Zwq4s4nxm9LWixpjaSbcuZtKuk3klZLeg0YWs1H9ACwu6Sv5szfHjgGuDsd\nP0HSHEnvS3pD0k+r+byfKdummuKQNErSwvSz+j9Jo9LytsCjwJ45R3e7pX/LO3PmP0nSS+ln9LSk\nvXOmLZP0fUnz0897gqQdq4i5h6Rpkt6VtErSn9IYyqZ3lfSQpJXp9Btzpn1b0svpNrwoqaji9yKt\nd4+kK9LhIySVSvqxpLeAP0hqL+mxdB1rJD0qqVPu30TSnel3YY2k+9PylyUdnVNvx3R6n6r+RrYl\nJ4JsOAm4F2gLTAI2ARcBHYADSXZQ365m/jOBnwK7khx1XLW1dSXtBkwGLknX+zowsJrl5BPjMcAA\noD9JgjsiLT8fOAooAvYDTq9qJRHxAXAfcHZO8RnAvIh4KR1fD4wA2gHHAxdJOq6a2MvUFMfbwLFA\nG+A84LeS+kbE2nQ9b+Qc3b2TO6OkfYE/AWOAjsAU4JGyZJk6HTgS+BLJ51TZkQ+AgJ8DuwM90/o/\nTdfTDPgLsBjoBnQh+TsiaThwafrZtAFOBt7N43MB6Ay0BvYE/oNkX/SHdLwr8DFwY079e4Ed0vh2\ny5l2N3BWTr3jgNKImJ9nHAYQEX41khdQChxRoeznwNM1zPdD4H/S4WZAAN3S8XuA/86pewLw4jbU\nPRf4R840ASuAc/Lctspi3D9n+gPAD9Ph6cConGnHJF/1Kpd9KMkObMd0/DlgTDX1fwdcmw7vlbts\n4JmybdqGOP4MfDcdPoJkh1bxb3lnOvwz4N6caU2At4DB6fgy4Iyc6dcDv8vzsz4VmJEOH5Qut2kl\n9aaWxVuhfLPvRc5344qcbfsI2KGaGEqAlelwF5IfBm0rqdcFeB/YKR1/CPh+If6/GvPLRwTZsDR3\nRNI+kv4i6S1J7wNXkvzyrspbOcMbSH7JbW3dPXLjiOS/dllVC8kzxrzWBSypJl6Av5PsTI6X9BWS\nI4wJObEcIOlvabPFWmBUJbFUpto4JB0n6bm0SeY9kqOHfJZbtuzy5UXEpySfZ6ecOnn93STtLmmy\nkpPj7wN35sTRhSQhVXaupAvwf3nGW9HbEbExJ4bWkm5Lm97eB56uEMOqSI6UNhMRS4HngZMl7Ury\nGd67jTFllhNBNlS8ZPFW4EVgr4hoA1xG8gu9kFaQNAcAIElsvtOqaHtiXEGy8yhT7eWtaVK6m6R5\n6BvAYxGxKqfKROB+oEtEtAVuyzOWKuOQ1JKkSeq/gC9ERDvgrznLreky0zdJmlDKlteE5PNdnkdc\nFV0D/Bvok37W5+TEsRToqsqvnloKfLliYURsSpfXKqd494rVKoxfAnQHBqYxHFZhPR0ktaki/rtI\nmoeGAdMj4q0q6lkVnAiyaWdgLfBB2tZc3fmB2vJnoFjS8Wm780UkbduFiHEy8D1JndITv/+Zxzx3\nk5yHOJdkx1Ixlncj4iNJ+5OcQ9jeOHYkafNeCXySnnM4PGf62yQ7v52rWfYJkg5NzwtcAqwjadba\nWjsDHwBrJXUhaYYr809gNfALSa0ktZR0YDrtNuBHkvor0SOdH2AuMELJCfNjgcF5xLABWJN+VuWX\n9Ka/+qcAN0tqJ6m5pINz5n0AGARcQHqC37aOE0E2/QD4JsmO41aSE8gFFRFvk/xiu55kx/Jl4AWS\nX461HeMtJO3X84EZJL+8a4pvMUkTw44kJ0dznQ/8l5Krrn5MerJ0e+KIiPeAi4EHSc5PnEqSLMum\nv0hyFFKaXhW0W4V4XyL5fG4hSSZDgRMi4uM8Y8t1OcmJ+7XAI+l6y9azieQE7L4kv8zfSGMlIiaQ\nHE1MImlaewDYJZ31QpKLFN4DTkuXW53rSS5mWA08CzxeYXrZCeFXSZLkmJwYPyA5N7Bn+m5bSekJ\nFrM6lTY1vAmcGhH/qO94rGGTdCWwZ0ScU9+xNEQ+IrA6I2loemi/I8nliR+T/Ao322ZpU9JIYFx9\nx9JQORFYXRoMvEbSlPE14KSIqKppyKxGks4naa56OCKere94Gio3DZmZZZyPCMzMMq7BdfbUoUOH\n6NatW32HYWbWoMyaNWtVRFR6yXaDSwTdunVj5syZ9R2GmVmDIqnKO+zdNGRmlnFOBGZmGedEYGaW\ncU4EZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGZmGedEYGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBmlnFO\nBGZmGedEYGaWcQVLBJJul/SOpBermC5JN0laLGmepOJCxWLWWI0fD926QZMmyfv48fUdUeF5m2t/\nmwt5RHAnMLSa6UcDPdLXaOCWAsaSOf5nafzbPH48jB4NS5ZARPI+enTj3m5vc4G2OSIK9gK6AS9W\nMe1WYHjO+CvAF2ta5oABA8Kqd889Ea1aRSRfm+TVqlVS3lhlcZu7dt18e8teXbvWd2SF423e9m0G\nZkYV+9X6PEfQCViaM74sLduCpNGSZkqauXLlyq1eUdZ+Kf7kJ7Bhw+ZlGzYk5Y1VFrf5jTe2rrwx\n8DbXXL4tGsQziyNiHDAOoKSkJLZm3rLDqrKdxJIlcNZZMGoU7LhjkhxyX02bbln2eZme77xLqngy\n6ZIl8ItfJL8nks91299rYxm1uczqtvn736/7v0FdLHePPWD58i23eY89YOlS+PTT5LP59NPNh2u7\nrC6X07YtvPfeltvcti1cc03l34GGrqpt3nPP2ltHfSaC5UCXnPHOaVmtquyXIkDz5jBy5Gdftk8+\n+Wy44mtbp23cWPvLrGlaPp/H9pA+e88drq33bZ23SZPkM6gs3ttuq/rza4yWL6/dnURD8N57MHZs\nfUdRd1q1gquvrr3l1WcieAS4QNJEYBCwNiJW1PZKqjp8Wr8efvOb2l5b/YuAe+6Bb38bPvzws/KW\nLeH3v4czz0zGt3ZH+3lX8cgPkn+WceNgxIiq58v95bk9SbhQyb2m6f/8Jzz8MLz7Luy6K5x4Ihxw\nQJIYpc+OHsqGt6estpazvcueNAl+9rPkqKdLl2T4jDMK/x2rTxMnwuWXJ9vctWuSBKr7Xm8tRdnx\ndS2TNAE4FOgAvA1cDjQHiIj/liTgdyRXFm0ARkbEzJqWW1JSEjNn1litXLdulTcbdO0KpaV5L6bB\nGT8++fX/xhvJr8Pa/uJ8HmVxm83yJWlWRJRUOq1QiaBQtjYRbOsvRTOzxqS6RNCkroOpayNGJDv9\nrl2Tw8uuXZ0EzMxyNYirhrbXiBHe8ZuZVaXRHxGYmVn1nAjMzDLOicDMLOOcCMzMMs6JwMws45wI\nzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzM\nMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLO\nicDMLOOcCMzMMs6JwMws4wqaCCQNlfSKpMWSxlYyfU9J0yS9IGmepGMKGY+ZmW2pYIlAUlPgZuBo\noCcwXFLPCtUuBSZHRH/gDOD3hYrHzMwqV8gjgoHA4oh4LSI2AhOBEyvUCaBNOtwWeLOA8ZiZWSUK\nmQg6AUtzxpelZbmuAM6StAx4DBhT2YIkjZY0U9LMlStXFiJWM7PMqu+TxcOBOyOiM3AM8CdJW8QU\nEeMioiQiSjp27FjnQZqZNWaFTATLgS45453TslzfAiYDRMQ/gRZAhwLGZGZmFRQyEcwAekjqLmkH\nkpPBj1So8wZwOICkfUkSgdt+zMzqUMESQURsAi4AngQWklwd9JKkKyWdkFb7AXCepLnABOCciIhC\nxWRmZltqVsiFR8RjJCeBc8suyxleABxYyBjMzKx69X2y2MzM6pkTgZlZxjkRmJllnBOBmVnGORGY\nmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJll\nnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWVcXolA0gOSjpXkxGFm\n1sjku2P/PXAmsEjSLyXtXcCYzMysDuWVCCJiSkSMAIqBUmCKpGcljZTUvJABmplZYeXd1COpPXAO\nMAp4AbiRJDE8VZDIzMysTjTLp5KkB4G9gT8Bx0fEinTSJEkzCxWcmZkVXl6JALgpIqZVNiEiSmox\nHjMzq2P5Ng31lNSubETSLpL+o0AxmZlZHco3EZwXEe+VjUTEGuC8mmaSNFTSK5IWSxpbRZ3TJS2Q\n9JKke/OMx8zMakm+TUNNJSkiAkBSU2CH6mZI69wMHAksA2ZIeiQiFuTU6QH8P+DAiFgjabdt2Qgz\nM9t2+R4RPEFyYvhwSYcDE9Ky6gwEFkfEaxGxEZgInFihznnAzekRBhHxTv6hm5lZbcj3iOA/gW8D\n56fjTwG31TBPJ2BpzvgyYFCFOl8BkPS/QFPgiojYIsFIGg2MBthzzz3zDNnMzPKRVyKIiE+BW9JX\nba+/B3Ao0BmYLqlP7vmIdP3jgHEAJSUlUcsxmJllWr73EfQA/gvoCbQoK4+IL1Uz23KgS85457Qs\n1zLguYj4GHhd0qskiWFGPnGZmdn2y/ccwR0kRwObgCHA3cA9NcwzA+ghqbukHYAzgEcq1HmI5GgA\nSR1ImopeyzMmMzOrBfkmgpYRMRVQRCyJiCuAY6ubISI2ARcATwILgckR8ZKkKyWdkFZ7ElgtaQEw\nDbgkIlZvy4aYmdm2yfdk8b/TLqgXSbqApImndU0zRcRjwGMVyi7LGQ7g++nLzMzqQb5HBBcBrYAL\ngQHAWcA3CxWUmZnVnRqPCNIbw4ZFxA+B9cDIgkdlZmZ1psYjgoj4BBhcB7GYmVk9yPccwQuSHgH+\nB/igrDAiHihIVGZmVmfyTQQtgNXAYTllATgRmJk1cPneWezzAmZmjVS+dxbfQXIEsJmIOLfWIzIz\nszqVb9PQn3OGWwAnAW/WfjhmZlbX8m0auj93XNIE4JmCRGRmZnUq3xvKKuoB+CEyZmaNQL7nCNax\n+TmCt0ieUWBmZg1cvk1DOxc6EDMzqx95NQ1JOklS25zxdpK+XriwzMysruR7juDyiFhbNpI+Qezy\nwoRkZmZ1Kd9EUFm9fC89NTOzz7F8E8FMSddL+nL6uh6YVcjAzMysbuSbCMYAG4FJwETgI+C7hQrK\nzMzqTr5XDX0AjC1wLGZmVg/yvWroKUntcsZ3kfRk4cIyM7O6km/TUIf0SiEAImINvrPYzKxRyDcR\nfCppz7IRSd2opDdSMzNrePK9BPQnwDOS/g4IOAgYXbCozMyszuR7svgJSSUkO/8XgIeADwsZmJmZ\n1Y18O50bBVwEdAbmAPsD/2TzR1eamVkDlO85gouA/YAlETEE6A+8V/0sZmbWEOSbCD6KiI8AJO0Y\nES8DexcuLDMzqyv5nixelt5H8BDwlKQ1wJLChWVmZnUl35PFJ6WDV0iaBrQFnihYVGZmVme2ugfR\niPh7IQIxM7P6sa3PLDYzs0bCicDMLOMKmggkDZX0iqTFkqrsvVTSKZIivWnNzMzqUMESgaSmwM3A\n0UBPYLiknpXU25nkPoXnChWLmZlVrZBHBAOBxRHxWkRsJHmgzYmV1LsKuIbkYTdmZlbHCpkIOgFL\nc8aXpWXlJBUDXSLiL9UtSNJoSTMlzVy5cmXtR2pmlmH1drJYUhPgeuAHNdWNiHERURIRJR07dix8\ncGZmGVLIRLAc6JIz3jktK7Mz0Bv4m6RSko7sHvEJYzOzulXIRDAD6CGpu6QdgDOAR8omRsTaiOgQ\nEd0iohvwL+CEiJhZwJjMzKyCgiWCiNgEXAA8CSwEJkfES5KulHRCodZrZmZbZ6u7mNgaEfEY8FiF\nssuqqHtoIWMxM7PK+c5iM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4\nJwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcC\nM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOz\njHMiMDPLuIImAklDJb0iabGksZVM/76kBZLmSZoqqWsh4zEzsy0VLBFIagrcDBwN9ASGS+pZodoL\nQElE9AXuA35VqHjMzKxyhTwiGAgsjojXImIjMBE4MbdCREyLiA3p6L+AzgWMx8zMKlHIRNAJWJoz\nviwtq8q3gMcrmyBptKSZkmauXLmyFkM0M7PPxcliSWcBJcC1lU2PiHERURIRJR07dqzb4MzMGrlm\nBVz2cqBLznjntGwzko4AfgIcEhH/3pYVffzxxyxbtoyPPvpomwK1xqlFixZ07tyZ5s2b13coZp9r\nhUwEM4AekrqTJIAzgDNzK0jqD9wKDI2Id7Z1RcuWLWPnnXemW7duSNqemK2RiAhWr17NsmXL6N69\ne32HY/a5VrCmoYjYBFwAPAksBCZHxEuSrpR0QlrtWqA18D+S5kh6ZFvW9dFHH9G+fXsnASsnifbt\n2/so0SwPhTwiICIeAx6rUHZZzvARtbUuJwGryN8Js/x8Lk4Wm5lZ/clkIhg/Hrp1gyZNkvfx47dv\neatXr6Zfv37069eP3XffnU6dOpWPb9y4Ma9ljBw5kldeeaXaOjfffDPjtzdYM7MKCto09Hk0fjyM\nHg0b0tvYlixJxgFGjNi2ZbZv3545c+YAcMUVV9C6dWt++MMfblYnIogImjSpPPfecccdNa7nu9/9\n7rYFWI82bdpEs2aZ+5qZNSiZOyL4yU8+SwJlNmxIymvb4sWL6dmzJyNGjKBXr16sWLGC0aNHU1JS\nQq9evbjyyivL6w4ePJg5c+awadMm2rVrx9ixYykqKuKAAw7gnXeSC6ouvfRSbrjhhvL6Y8eOZeDA\ngey99948++yzAHzwwQeccsop9OzZk1NPPZWSkpLyJJXr8ssvZ7/99qN379585zvfISIAePXVVzns\nsMMoKiqiuLiY0tJSAH7xi1/Qp08fioqK+En6YZXFDPDWW2+x1157AXDbbbfx9a9/nSFDhvC1r32N\n999/n8MOO4zi4mL69u3Ln//85/I47rjjDvr27UtRUREjR45k7dq1fOlLX2LTpk0ArFmzZrNxM6t9\nmUsEb7yxdeXb6+WXX+biiy9mwYIFdOrUiV/+8pfMnDmTuXPn8tRTT7FgwYIt5lm7di2HHHIIc+fO\n5YADDuD222+vdNkRwfPPP8+1115bnlR++9vfsvvuu7NgwQJ++tOf8sILL1Q670UXXcSMGTOYP38+\na9eu5YknngBg+PDhXHzxxcydO5dnn32W3XbbjUcffZTHH3+c559/nrlz5/KDH/ygxu1+4YUXeOCB\nB5g6dSotW7bkoYceYvbs2UyZMoWLL74YgLlz53LNNdfwt7/9jblz53LdddfRtm1bDjzwwPJ4JkyY\nwGmnneajCrMCylwi2HPPrSvfXl/+8pcpKSkpH58wYQLFxcUUFxezcOHCShNBy5YtOfroowEYMGBA\n+a/yik4++eQt6jzzzDOcccYZABQVFdGrV69K5506dSoDBw6kqKiIv//977z00kusWbOGVatWcfzx\nxwPJDVmtWrViypQpnHvuubRs2RKAXXfdtcbtPuqoo9hll12AJGGNHTuWvn37ctRRR7F06VJWrVrF\n008/zbBhw8qXV/Y+atSo8qayO+64g5EjR9a4PjPbdplLBFdfDa1abV7WqlVSXgg77bRT+fCiRYu4\n8cYbefrpp5k3bx5Dhw6t9Dr3HXbYoXy4adOmVTaL7LjjjjXWqcyGDRu44IILePDBB5k3bx7nnnvu\nNl1v36xZMz799FOALebP3e67776btWvXMnv2bObMmUOHDh2qXd8hhxzCq6++yrRp02jevDn77LPP\nVsdmZvnLXCIYMQLGjYOuXUFK3seN2/YTxVvj/fffZ+edd6ZNmzasWLGCJ598stbXceCBBzJ58mQA\n5s+fX+kRx4cffkiTJk3o0KED69at4/777wdgl112oWPHjjz66KNAsnPfsGEDRx55JLfffjsffvgh\nAO+++y4A3bp1Y9asWQDcd999Vca0du1adtttN5o1a8ZTTz3F8uVJTyOHHXYYkyZNKl9e2TvAWWed\nxYgRI3w0YFYHMpcIINnpl5bCp58m73WRBACKi4vp2bMn++yzD2effTYHHnhgra9jzJgxLF++nJ49\ne/Kzn/2Mnj170rZt283qtG/fnm9+85v07NmTo48+mkGDBpVPGz9+PNdddx19+/Zl8ODBrFy5kuOO\nO46hQ4dSUlJCv379+M1vfgPAJZdcwo033khxcTFr1qypMqZvfOMbPPvss/Tp04eJEyfSo0cPIGm6\n+tGPfsTBBx9Mv379uOSSS8rnGTFiBGvXrmXYsGG1+fGYWSVUdrVIQ1FSUhIzZ87crGzhwoXsu+++\n9RTR58umTZvYtGkTLVq0YNGiRRx11FEsWrSowZ1snThxIk8++WRel9VWx98Ns4SkWRFRUtm0hrV3\nsBqtX7+eww8/nE2bNhER3HrrrQ0uCZx//vlMmTKl/MohMyushrWHsBq1a9euvN2+obrlllvqOwSz\nTMnkOQIzM/uME4GZWcY5EZiZZZwTgZlZxjkR1IIhQ4ZscXPYDTfcwPnnn1/tfK1btwbgzTff5NRT\nT620zqGHHkrFy2UruuGGG9iQ05PeMcccw3vvvZdP6GZmTgS1Yfjw4UycOHGzsokTJzJ8+PC85t9j\njz2qvTO3JhUTwWOPPUa7du22eXl1LSLKu6ows7rX6BLB974Hhx5au6/vfa/6dZ566qn85S9/KX8I\nTWlpKW+++SYHHXRQ+XX9xcXF9OnTh4cffniL+UtLS+nduzeQdP9wxhlnsO+++3LSSSeVd+sAyfX1\nZV1YX3755QDcdNNNvPnmmwwZMoQhQ4YASdcPq1atAuD666+nd+/e9O7du7wL69LSUvbdd1/OO+88\nevXqxVFHHbXZeso8+uijDBo0iP79+3PEEUfw9ttvA8m9CiNHjqRPnz707du3vIuKJ554guLiYoqK\nijj88MOB5PkMv/71r8uX2bt3b0pLSyktLWXvvffm7LPPpnfv3ixdurTS7QOYMWMGX/3qVykqKmLg\nwIGsW7eOgw8+eLPutQcPHszcuXOr/0OZWaV8H0Et2HXXXRk4cCCPP/44J554IhMnTuT0009HEi1a\ntODBBx+kTZs2rFq1iv33358TTjihyufp3nLLLbRq1YqFCxcyb948iouLy6ddffXV7LrrrnzyyScc\nfvjhzJs3jwsvvJDrr7+eadOm0aFDh82WNWvWLO644w6ee+45IoJBgwZxyCGHsMsuu7Bo0SImTJjA\nH/7wB04//XTuv/9+zjrrrM3mHzx4MP/617+QxG233cavfvUrrrvuOq666iratm3L/PnzgeSZAStX\nruS8885j+vTpdO/efbN+g6qyaNEi7rrrLvbff/8qt2+fffZh2LBhTJo0if3224/333+fli1b8q1v\nfYs777yTG264gVdffZWPPvqIoqKirfq7mVmi0SWC9EdvnStrHipLBH/84x+BpNnjxz/+MdOnT6dJ\nkyYsX76ct99+m913373S5UyfPp0LL7wQgL59+9K3b9/yaZMnT2bcuHFs2rSJFStWsGDBgs2mV/TM\nM89w0kknlfcEevLJJ/OPf/yDE044ge7du9OvXz+g6q6uly1bxrBhw1ixYgUbN26ke/fuAEyZMmWz\nprBddtmFRx99lIMPPri8Tj5dVXft2rU8CVS1fZL44he/yH777QdAmzZtADjttNO46qqruPbaa7n9\n9ts555xzalyfmVWu0TUN1ZcTTzyRqVOnMnv2bDZs2MCAAQOApBO3lStXMmvWLObMmcMXvvCFbery\n+fXXX+fXv/41U6dOZd68eRx77LHbtJwyZV1YQ9XdWI8ZM4YLLriA+fPnc+utt253V9WweXfVuV1V\nb+32tWrViiOPPJKHH36YyZMnM6Kueg40a4ScCGpJ69atGTJkCOeee+5mJ4nLumBu3rw506ZNY8mS\nJdUu5+CDD+bee+8F4MUXX2TevHlA0oX1TjvtRNu2bXn77bd5/PHHy+fZeeedWbdu3RbLOuigg3jo\noYfYsGEDH3zwAQ8++CAHHXRQ3tu0du1aOnXqBMBdd91VXn7kkUdy8803l4+vWbOG/fffn+nTp/P6\n668Dm3dVPXv2bABmz55dPr2iqrZv7733ZsWKFcyYMQOAdevWlSetUaNGceGFF7LffvuVPwTHzLae\nE0EtGj58OHPnzt0sEYwYMYKZM2fSp08f7r777hofsnL++eezfv169t13Xy677LLyI4uioiL69+/P\nPvvsw5lnnrlZF9ajR49m6NCh5SeLyxQXF3POOecwcOBABg0axKhRo+jfv3/e23PFFVdw2mmnMWDA\ngM3OP1x66aWsWbOG3r17U0c3l/gAAAZwSURBVFRUxLRp0+jYsSPjxo3j5JNPpqioqLz76FNOOYV3\n332XXr168bvf/Y6vfOUrla6rqu3bYYcdmDRpEmPGjKGoqIgjjzyy/EhhwIABtGnTxs8sMNtO7oba\nGqw333yTQw89lJdffpkmTSr/TePvhlmium6ofURgDdLdd9/NoEGDuPrqq6tMAmaWn0Z31ZBlw9ln\nn83ZZ59d32GYNQqN5qdUQ2vissLzd8IsP40iEbRo0YLVq1f7H9/KRQSrV6+mRYsW9R2K2edeo2ga\n6ty5M8uWLWPlypX1HYp9jrRo0YLOnTvXdxhmn3uNIhE0b968/I5WMzPbOgVtGpI0VNIrkhZLGlvJ\n9B0lTUqnPyepWyHjMTOzLRUsEUhqCtwMHA30BIZL6lmh2reANRGxF/Ab4JpCxWNmZpUr5BHBQGBx\nRLwWERuBicCJFeqcCJT1XXAfcLiq6pbTzMwKopDnCDoBS3PGlwGDqqoTEZskrQXaA6tyK0kaDYxO\nR9dLemUbY+pQcdkZ4G3OBm9zNmzPNnetakKDOFkcEeOAcdu7HEkzq7rFurHyNmeDtzkbCrXNhWwa\nWg50yRnvnJZVWkdSM6AtsLqAMZmZWQWFTAQzgB6SukvaATgDeKRCnUeAb6bDpwJPh+8KMzOrUwVr\nGkrb/C8AngSaArdHxEuSrgRmRsQjwB+BP0laDLxLkiwKabublxogb3M2eJuzoSDb3OC6oTYzs9rV\nKPoaMjOzbedEYGaWcZlIBJJul/SOpBfrO5a6IqmLpGmSFkh6SdJF9R1ToUlqIel5SXPTbf5ZfcdU\nFyQ1lfSCpD/Xdyx1QVKppPmS5kiaWfMcDZ+kdpLuk/SypIWSDqjV5WfhHIGkg4H1wN0R0bu+46kL\nkr4IfDEiZkvaGZgFfD0iFtRzaAWT3pW+U0Ssl9QceAa4KCL+Vc+hFZSk7wMlQJuIOK6+4yk0SaVA\nSURk5mYySXcB/4iI29KrMFtFxHu1tfxMHBFExHSSq5IyIyJWRMTsdHgdsJDkTu5GKxLr09Hm6atR\n/9KR1Bk4FritvmOxwpDUFjiY5CpLImJjbSYByEgiyLq0V9f+wHP1G0nhpc0kc4B3gKciorFv8w3A\nj4BP6zuQOhTAXyXNSrufaey6AyuBO9ImwNsk7VSbK3AiaOQktQbuB74XEe/XdzyFFhGfREQ/kjvZ\nB0pqtE2Bko4D3omIWfUdSx0bHBHFJD0bfzdt+m3MmgHFwC0R0R/4ANiiW//t4UTQiKXt5PcD4yPi\ngfqOpy6lh87TgKH1HUsBHQickLaZTwQOk3RP/YZUeBGxPH1/B3iQpKfjxmwZsCzn6PY+ksRQa5wI\nGqn0xOkfgYURcX19x1MXJHWU1C4dbgkcCbxcv1EVTkT8v4joHBHdSO7KfzoizqrnsApK0k7pxQ+k\nzSNHAY36asCIeAtYKmnvtOhwoFYv+mgQvY9uL0kTgEOBDpKWAZdHxB/rN6qCOxD4BjA/bTMH+HFE\nPFaPMRXaF4G70ociNQEmR0QmLqnMkC8AD6aPLWkG3BsRT9RvSHViDDA+vWLoNWBkbS48E5ePmplZ\n1dw0ZGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGYpSZ+kPVqWvWrt7k1J3bLU+601LJm4j8As\nTx+m3VOYZYqPCMxqkPZ//6u0D/znJe2VlneT9LSkeZKmStozLf+CpAfT5yLMlfTVdFFNJf0hfVbC\nX9O7n5F0YfrciHmSJtbTZlqGORGYfaZlhaahYTnT1kZEH+B3JD1+AvwWuCsi+gLjgZvS8puAv0dE\nEUmfMC+l5T2AmyOiF/AecEpaPhbony7nO4XaOLOq+M5is5Sk9RHRupLyUuCwiHgt7cjvrYhoL2kV\nycN/Pk7LV0REB0krgc4R8e+cZXQj6Ra7Rzr+n0DziPi5pCdIHpz0EPBQzjMVzOqEjwjM8hNVDG+N\nf+cMf8Jn5+iOBW4mOXqYIcnn7qxOORGY5WdYzvs/0+FnSXr9BBgB/CMdngqcD+UPymlb1UIlNQG6\nRMQ04D+BtsAWRyVmheRfHmafaZnTUyvAExFRdgnpLpLmkfyqH56WjSF5atQlJE+QKusR8iJgnKRv\nkfzyPx9YUcU6mwL3pMlCwE21/RhCs5r4HIFZDbL4sHTLFjcNmZllnI8IzMwyzkcEZmYZ50RgZpZx\nTgRmZhnnRGBmlnFOBGZmGff/AdTOYjzxo2ZvAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5gU9Z3v8feHiyKCsAJKBGRYY0S8\ncHEOWbwQjYmLNzgqKkjUaAxZj9dEPSHGuNHENdnkUVbDcUNMjFlxkGi8rItrEiVe1ngZFBAYXFgy\n6iDiBBDlojD4PX9UMTZDzzAzTE8zU5/X8/RD16+qfv2tHu1P16+qqxQRmJlZdnUodgFmZlZcDgIz\ns4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4G1GEkdJa2XdGBLLltMkj4rqSDnWNftW9LvJU0qRB2S\nvifpX5u7vrVvDoIMSz+Itz0+kbQpZzrvB1JDImJrRHSLiLdactndlaQ/SroxT/tZklZI6tiU/iLi\npIiY0QJ1fUlSZZ2+fxAR/7Crfe/kNUPSNTurJW1/XtJXc6YHS3pQ0mpJ6yTNl3S1JH9GtQK/yRmW\nfhB3i4huwFvA6TltO3wgSerU+lXu1u4Fzs/Tfj5wX0RsbeV6iulCYA1wQVNXlHQw8CKwHDg8InoA\nE4BRQNeWLNLycxBYvST9UNIDksokfQh8RdIoSS9Kel/SSkl3SOqcLt8p/VZYkk7fl85/QtKHkv4s\naVBTl03nnyzpv9Nvi3dK+q/cb5R16m5Mjd+QtEzSWkl35KzbUdLt6TfT5cCYBt6i3wF9JR2ds34v\n4BTgN+n0WEnzJH0g6S1J32vg/a79lryzOiRdIqkifa/+R9IlaXsP4N+BA3P27vZL/5a/zln/DEmL\n0vfoaUmH5MyrkvQtSa+n73eZpD0bqLs7cCbwf4AhkoY18J7l8wPgmYj4vxGxEiAiKiLi3IhY38S+\nrBkcBLYzZwD3Az2AB4Aa4CqgN3AMyQfUNxpY/zzge8C+JHsdP2jqspL2A2YB16Wv+xdgZAP9NKbG\nU4CjgOEkAfeltP1S4CRgKPC/gHPqe5GI2AA8yPbfgicACyJiUTq9HpgE9AROB66SdFoDtW+zszpW\nAacC+wBfB+6UdGRErEtf562cvbv3cleUdCjwb8AVQB/gj8Bj28IydQ7wZeBvSd6nfHs+24wH1gK/\nTfu6sBHbl+tLJO+jFYmDwHbm+Yj494j4JCI2RcQrEfFSRNRExHJgOvCFBtZ/MCLKI2ILMANo6Nti\nfcueBsyLiEfTebcDf62vk0bWeGtErIuISuBPOa91DnB7RFRFxGrgRw3UC8nw0Dk535gvSNu21fJ0\nRCxK37/5wMw8teTTYB3p32R5JJ4GngKOa0S/kITVY2ltW9K+ewCfz1lmakS8m7724zT8d7sQmBkR\nn5B8aTivicOI+wIrm7C8tTAHge3M27kT6UG9/5D0rqQPgJtJvnnX592c5xuBbs1Y9oDcOiK5UmJV\nfZ00ssZGvRbwZgP1AjwDfACcLulzJHsYZTm1jJL0J0nVktYBl+SpJZ8G65B0mqSXJK2R9D7J3kNj\n+t3Wd21/6Qd4FdAvZ5lG/d3Sob3RJMEN8HC67LahrBqg8w4rJm1b0udrgM80snYrAAeB7UzdUxZ/\nDiwEPhsR+wA3AipwDSuB/tsmJIntP7Tq2pUaVwIDcqYbPL01DaXfkOwJnA/MjojcvZWZwEPAgPQg\n6N2NrKXeOiTtRTKUciuwf0T0BH6f0+/OTjN9BxiY018Hkvd3RSPqquuC9HWfkPQusAzYg0+Hh94C\n9pNUe9A3/fsN5NMw+iNwVjNe21qIg8CaqjuwDtiQjjU3dHygpTwOjJB0ejrkcBXJ2HYhapwFXC2p\nX3rg99uNWOc3JN+ALyZnWCinljUR8ZGkvyMZltnVOvYk+bCtBramxxxOzJm/CuidHsStr++xko5P\njwtcB3wIvNTI2nJdQBK0w3Ie55LsIf0NyfGc14AfSdpbUhdgCrABeCXt40bgeEm3SuoLIOlzku6X\n1NAepLUQB4E11TUk3/Y+JPnm/UChXzAiVpF8uNwGrAYOIvlw+bgANd5FMt7+OskH1U4PYkbEMuBl\nkg/o/6gz+1LgViVnXV1P8iG8S3VExPvAN0mGYdaQHKx9PGf+QpK9kMr0rKD96tS7iOT9uYskTMYA\nY9PjBY0m6ViSYaZp6fGEdyPi3bSuSuDcdI9pPMke3HKSIagvAKdGxMdpPf9Ncqro54DF6VDXLJJT\nSjc2pSZrHvnGNNbWKPmh1jvA+Ih4rtj1mLV13iOwNkHSGEk907NzvkdyoPHlIpdl1i44CKytOJZk\naKEa+HvgjG1DC2a2azw0ZGaWcd4jMDPLuDZ3EbHevXtHSUlJscswM2tT5s6d+9eIyHvadZsLgpKS\nEsrLy4tdhplZmyKp3l/Je2jIzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZ\nZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMK1gQSPqVpPckLaxnviTdIWmZ\npAWSRhSqliyaMQNKSqBDh+TfGTOKXVHheZu9ze1Vwbc5IgryAEYDI4CF9cw/BXgCEPB3wEuN6feo\no44Ka9h990V07RoBnz66dk3a2ytvs7e5vWqpbQbKo57P1YLevF5SCfB4RByeZ97PgT9FRFk6/QZw\nfESsbKjP0tLS8B3KGlZSAm/muRfRwIFQWdna1bQOb/OnvM3tS0tts6S5EVGab14xb1XZD3g7Z7oq\nbdshCCRNBiYDHHjggU1+oRkz4MorYc0a6NgR9t8f+vaFvfbK/+jatf55jVmmc+fmvSEt5a23mtbe\nHnibd97eHnibd97eHG3insURMR2YDskeQVPWnTEDJk+GjRuT6a1bYdUq2G8/2HNP2LABqqth06Yd\nH5980rx6O3ZsuVBpbPBIn77+gQfm/wbRjAxtM7zN27e3V97m7dtbSjGDYAUwIGe6f9rWor773U9D\nYJutW2HtWnjttfrXi4AtW7YPho0b8wdG3cfOlluzJv9yW7c2bxs7dNg+MLZsSYIhd9Rv2zKjR3+6\nfU35tznrtGafmzbtuM1S8h7/7d/W/7p1nzd2ud1h/c2byau6Gnr2zD8vn10ZHW7tdWtq8revWgX7\n7LN9W+6Xo9znTZlXqGWb0k++/7a7doVbbqHFFDMIHgMulzQT+DywbmfHB5qjubtVEuyxR/Lo0aOl\nq8qvbvDsSvgsXQoVFfDRR8mez6BByXDYtm1rzr+7sm5r9FlZCfPnJ3t5e+8Nw4Yl213f/4DN+Z92\nd1t/yRL485/hww+he3c4+mgYPJgmq/v6u/O6FRXw/POfbvOxx+64zc0J2tZatjn9/M//QHl58t/2\nwIFJCEyaRIspWBBIKgOOB3pLqgL+EegMEBH/CswmOXNoGbARuKgQdbSlXcnOnZNH3W82ZmaFVLAg\niIiJO5kfwGWFev1tbrll+2ME0PK7VWZmbVm7/2XxpEkwfXqyOyUl/06f3rK7VWZmbVmbOGtoV02a\n5A9+M7P6tPs9AjMza5iDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWWc\ng8DMLOMcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAz\nyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcYVNAgkjZH0\nhqRlkqbkmX+gpDmSXpO0QNIphazHzMx2VLAgkNQRmAacDAwBJkoaUmexG4BZETEcmAD8v0LVY2Zm\n+RVyj2AksCwilkfEZmAmMK7OMgHskz7vAbxTwHrMzCyPQgZBP+DtnOmqtC3X94GvSKoCZgNX5OtI\n0mRJ5ZLKq6urC1GrmVlmFftg8UTg1xHRHzgF+DdJO9QUEdMjojQiSvv06dPqRZqZtWeFDIIVwICc\n6f5pW66vAbMAIuLPQBegdwFrMjOzOgoZBK8AB0saJGkPkoPBj9VZ5i3gRABJh5IEgcd+zMxaUcGC\nICJqgMuBJ4EKkrODFkm6WdLYdLFrgK9Lmg+UAV+NiChUTWZmtqNOhew8ImaTHATObbsx5/li4JhC\n1mBmZg0r9sFiMzMrMgeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgH\ngZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaW\ncQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllXEGD\nQNIYSW9IWiZpSj3LnCNpsaRFku4vZD1mZrajToXqWFJHYBrwZaAKeEXSYxGxOGeZg4HvAMdExFpJ\n+xWqHjMzy6+QewQjgWURsTwiNgMzgXF1lvk6MC0i1gJExHsFrMfMzPIoZBD0A97Oma5K23J9Dvic\npP+S9KKkMfk6kjRZUrmk8urq6gKVa2aWTcU+WNwJOBg4HpgI/EJSz7oLRcT0iCiNiNI+ffq0colm\nZu1bvUEg6e8ljc/TPl7SlxvR9wpgQM50/7QtVxXwWERsiYi/AP9NEgxmZtZKGtojuBF4Jk/7n4Cb\nG9H3K8DBkgZJ2gOYADxWZ5lHSPYGkNSbZKhoeSP6NjOzFtJQEOwZETsMyEfEX4G9d9ZxRNQAlwNP\nAhXArIhYJOlmSWPTxZ4EVktaDMwBrouI1U3dCDMza76GTh/dR1Kn9AO9lqTOwF6N6TwiZgOz67Td\nmPM8gG+lDzMzK4KG9gh+R3Lwtvbbv6RuwL+m88zMrB1oKAhuAFYBb0qaK+lV4C9AdTrPzMzagXqH\nhtIhoSmSbgI+mzYvi4hNrVKZmZm1inqDQNKZdZoC6ClpXkR8WNiyzMystTR0sPj0PG37AkdK+lpE\nPF2gmszMrBU1NDR0Ub52SQOBWcDnC1WUmZm1niZfYiIi3gQ6F6AWMzMrgiYHgaTBwMcFqMXMzIqg\noYPF/05ygDjXvsBngK8UsigzM2s9DR0s/mmd6QDWkITBV4A/F6ooMzNrPQ0dLK694Jyk4cB5wNkk\nPyp7qPClmZlZa2hoaOhzJPcImAj8FXgAUESc0Eq1mZlZK2hoaGgJ8BxwWkQsA5D0zVapyszMWk1D\nZw2dCawE5kj6haQTAbVOWWZm1lrqDYKIeCQiJgCDSe4VcDWwn6S7JJ3UWgWamVlh7fR3BBGxISLu\nj4jTSW43+Rrw7YJXZmZmraJJPyiLiLXpjeRPLFRBZmbWupr8y2IzM2tfHARmZhnnIDAzyzgHgZlZ\nxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQUN\nAkljJL0haZmkKQ0sd5akkFRayHrMzGxHBQsCSR2BacDJwBBgoqQheZbrDlwFvFSoWszMrH6F3CMY\nCSyLiOURsRmYCYzLs9wPgB8DHxWwFjMzq0chg6Af8HbOdFXaVkvSCGBARPxHQx1JmiypXFJ5dXV1\ny1dqZpZhRTtYLKkDcBtwzc6WTW+PWRoRpX369Cl8cWZmGVLIIFgBDMiZ7p+2bdMdOBz4k6RK4O+A\nx3zA2MysdRUyCF4BDpY0SNIewATgsW0zI2JdRPSOiJKIKAFeBMZGRHkBazIzszoKFgQRUQNcDjwJ\nVACzImKRpJsljS3U65qZWdN0KmTnETEbmF2n7cZ6lj2+kLWYmVl+/mWxmVnGOQjMzDLOQWBmlnEO\nAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws\n4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeB\nmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjChoEksZIekPSMklT8sz/lqTFkhZIekrSwELW\nY2ZmOypYEEjqCEwDTgaGABMlDamz2GtAaUQcCTwI/HOh6jEzs/wKuUcwElgWEcsjYjMwExiXu0BE\nzImIjenki0D/AtZjZmZ5FDII+gFv50xXpW31+RrwRL4ZkiZLKpdUXl1d3YIlmpnZbnGwWNJXgFLg\nJ/nmR8T0iCiNiNI+ffq0bnFmZu1cpwL2vQIYkDPdP23bjqQvAd8FvhARHzfnhbZs2UJVVRUfffRR\nswq14ujSpQv9+/enc+fOxS7FLNMKGQSvAAdLGkQSABOA83IXkDQc+DkwJiLea+4LVVVV0b17d0pK\nSpC0KzVbK4kIVq9eTVVVFYMGDSp2OWaZVrChoYioAS4HngQqgFkRsUjSzZLGpov9BOgG/FbSPEmP\nNee1PvroI3r16uUQaEMk0atXL+/Fme0GCrlHQETMBmbXabsx5/mXWuq1HAJtj/9mZruH3eJgsZmZ\nFU8mg2DGDCgpgQ4dkn9nzNi1/lavXs2wYcMYNmwYffv2pV+/frXTmzdvblQfF110EW+88UaDy0yb\nNo0Zu1psjlWrVtGpUyfuvvvu2raamhp69uy53XJ33303V199de30r3/9aw4//HCOOOIIRowYwe23\n395iNZlZ6yvo0NDuaMYMmDwZNqY/Y3vzzWQaYNKk5vXZq1cv5s2bB8D3v/99unXrxrXXXrvdMhFB\nRNChQ/7sveeee3b6OpdddlnzCqzHrFmzGDVqFGVlZVxyySWNWufxxx/nZz/7GX/84x/p27cvH330\nEffdd1+L1mVmrStzewTf/e6nIbDNxo1Je0tbtmwZQ4YMYdKkSRx22GGsXLmSyZMnU1paymGHHcbN\nN99cu+yxxx7LvHnzar+RT5kyhaFDhzJq1Cjeey85oeqGG25g6tSptctPmTKFkSNHcsghh/DCCy8A\nsGHDBs466yyGDBnC+PHjKS0trQ2pusrKypg6dSrLly9n5cqVjdqmf/qnf+K2226jb9++QHIKaGND\nxMx2T5kLgrfealr7rlqyZAnf/OY3Wbx4Mf369eNHP/oR5eXlzJ8/nz/84Q8sXrx4h3XWrVvHF77w\nBebPn8+oUaP41a9+lbfviODll1/mJz/5SW2o3HnnnfTt25fFixfzve99j9deey3vupWVlaxZs4aj\njjqKs88+m1mzZjVqexYtWsRRRx3VyK03s7Ygc0Fw4IFNa99VBx10EKWlpbXTZWVljBgxghEjRlBR\nUZE3CPbaay9OPvlkAI466igqKyvz9n3mmWfusMzzzz/PhAkTABg6dCiHHXZY3nVnzpzJueeeC8CE\nCRMoKysD6j+Tx2f4mLVfmQuCW26Brl23b+vaNWkvhL333rv2+dKlS/mXf/kXnn76aRYsWMCYMWPy\nnke/xx571D7v2LEjNTU1efvec889d7pMfcrKyrj77rspKSnhzDPP5NVXX2X58uV07NiRDh06bNff\nmjVr6N27NwBDhgxh7ty5TXotM9u9ZS4IJk2C6dNh4ECQkn+nT2/+geKm+OCDD+jevTv77LMPK1eu\n5Mknn2zx1zjmmGNqh3lef/31vHscixcvpqamhhUrVlBZWUllZSXXXXcdM2fOBGD06NHcf//9AGzc\nuJHf/va3nHDCCQB85zvf4dprr2XVqlUAfPzxx/zyl79s8e0ws9aTubOGIPnQb40P/rpGjBjBkCFD\nGDx4MAMHDuSYY45p8de44ooruOCCCxgyZEjto0ePHtstU1ZWxhlnnLFd21lnncWFF17I9ddfz513\n3sk3vvENbrvtNj755BMuvvhijj76aADGjh1LdXU1X/ziF4FkyOjrX/96i2+HmbUeRUSxa2iS0tLS\nKC8v366toqKCQw89tEgV7V5qamqoqamhS5cuLF26lJNOOomlS5fSqdPumfn+25m1DklzI6I037zd\n89PBmm39+vWceOKJ1NTUEBH8/Oc/321DwMx2D/6EaGd69uzpg7lm1iSZO1hsZmbbcxCYmWWcg8DM\nLOMcBGZmGecgaAEnnHDCDj8Omzp1KpdeemmD63Xr1g2Ad955h/Hjx+dd5vjjj6fu6bJ1TZ06lY05\nV9I75ZRTeP/99xtTeqMMGzas9rIV9dVVWVnJ4YcfXjv98ssvM3r0aA455BCGDx/OJZdcsl2NZrb7\ncBC0gIkTJ9b+KnebmTNnMnHixEatf8ABB/Dggw82+/XrBsHs2bN3uKdAc1VUVLB161aee+45NmzY\n0Kh1Vq1axdlnn82Pf/xj3njjDV577TXGjBnDhx9+2CI1mVnLanenj159NdRz1eVmGzYM0qs/5zV+\n/HhuuOEGNm/ezB577EFlZSXvvPMOxx13HOvXr2fcuHGsXbuWLVu28MMf/pBx48Ztt35lZSWnnXYa\nCxcuZNOmTVx00UXMnz+fwYMHs2nTptrlLr30Ul555RU2bdrE+PHjuemmm7jjjjt45513OOGEE+jd\nuzdz5syhpKSE8vJyevfuzW233VZ79dJLLrmEq6++msrKSk4++WSOPfZYXnjhBfr168ejjz7KXnvt\ntcO2lZWVcf7551NRUcGjjz7Keeedt9P3a9q0aVx44YWMGjVqu/fIzHZP3iNoAfvuuy8jR47kiSee\nAJK9gXPOOQdJdOnShYcffphXX32VOXPmcM0119DQr7nvuusuunbtSkVFBTfddNN2vwm45ZZbKC8v\nZ8GCBTzzzDMsWLCAK6+8kgMOOIA5c+YwZ86c7fqaO3cu99xzDy+99BIvvvgiv/jFL2ovS7106VIu\nu+wyFi1aRM+ePXnooYfy1vPAAw8wYcIEJk6cWHuF0p1ZuHChL1Vt1oa0uz2Chr65F9K24aFx48Yx\nc+bM2guxRQTXX389zz77LB06dGDFihWsWrWq9sYudT377LNceeWVABx55JEceeSRtfNmzZrF9OnT\nqampYeXKlSxevHi7+XU9//zznHHGGbVXQD3zzDN57rnnGDt2LIMGDWLYsGFA/Ze63rZXceCBB9Kv\nXz8uvvhi1qxZw7777pv3stS+VLVZ2+Q9ghYybtw4nnrqKV599VU2btxY+414xowZVFdXM3fuXObN\nm8f++++f99LTO/OXv/yFn/70pzz11FMsWLCAU089tVn9bLPtEtZQ/2Wsy8rKWLJkCSUlJRx00EF8\n8MEHtXsOvXr1Yu3atbXL5l6q+rDDDvOvm83aEAdBC+nWrRsnnHACF1988XYHidetW8d+++1H586d\nmTNnDm+++WaD/eReAnrhwoUsWLAASC5hvffee9OjRw9WrVpVOwwF0L1797wHYo877jgeeeQRNm7c\nyIYNG3j44Yc57rjjGrU9n3zyCbNmzeL111+vvVT1o48+Wjs8dPzxx3PffffVDnPde++9tZeqvvzy\ny7n33nt56aWXavv73e9+V3vpajPbvbS7oaFimjhxImecccZ2ZxBNmjSJ008/nSOOOILS0lIGDx7c\nYB+XXnopF110EYceeiiHHnpo7Z7F0KFDGT58OIMHD2bAgAHbXcJ68uTJjBkzpvZYwTYjRozgq1/9\nKiNHjgSSg8XDhw+v945nuZ577jn69evHAQccUNs2evRoFi9eXHvv5SVLljB06FAkUVpayq233grA\n/vvvz8yZM7n22mt577336NChA6NHj2bMmDE7fxPNrNX5MtRWVP7bmbWOhi5D7aEhM7OMcxCYmWVc\nuwmCtjbEZf6bme0u2kUQdOnShdWrV/uDpQ2JCFavXk2XLl2KXYpZ5rWLs4b69+9PVVUV1dXVxS7F\nmqBLly7079+/2GWYZV67CILOnTszaNCgYpdhZtYmFXRoSNIYSW9IWiZpSp75e0p6IJ3/kqSSQtZj\nZmY7KlgQSOoITANOBoYAEyUNqbPY14C1EfFZ4Hbgx4Wqx8zM8ivkHsFIYFlELI+IzcBMYFydZcYB\n96bPHwROlK9cZmbWqgp5jKAf8HbOdBXw+fqWiYgaSeuAXsBfcxeSNBmYnE6ul/RGM2vqXbfvDPA2\nZ4O3ORt2ZZsH1jejTRwsjojpwPRd7UdSeX0/sW6vvM3Z4G3OhkJtcyGHhlYAA3Km+6dteZeR1Ano\nAawuYE1mZlZHIYPgFeBgSYMk7QFMAB6rs8xjwIXp8/HA0+FfhZmZtaqCDQ2lY/6XA08CHYFfRcQi\nSTcD5RHxGPBL4N8kLQPWkIRFIe3y8FIb5G3OBm9zNhRkm9vcZajNzKxltYtrDZmZWfM5CMzMMi4T\nQSDpV5Lek7Sw2LW0FkkDJM2RtFjSIklXFbumQpPURdLLkuan23xTsWtqDZI6SnpN0uPFrqU1SKqU\n9LqkeZLKd75G2yepp6QHJS2RVCFpVIv2n4VjBJJGA+uB30TE4cWupzVI+gzwmYh4VVJ3YC7wvyNi\ncZFLK5j0V+l7R8R6SZ2B54GrIuLFIpdWUJK+BZQC+0TEacWup9AkVQKlEZGZH5NJuhd4LiLuTs/C\n7BoR77dU/5nYI4iIZ0nOSsqMiFgZEa+mzz8EKkh+yd1uRWJ9Otk5fbTrbzqS+gOnAncXuxYrDEk9\ngNEkZ1kSEZtbMgQgI0GQdelVXYcDLxW3ksJLh0nmAe8Bf4iI9r7NU4H/C3xS7EJaUQC/lzQ3vfxM\nezcIqAbuSYcA75a0d0u+gIOgnZPUDXgIuDoiPih2PYUWEVsjYhjJL9lHSmq3Q4GSTgPei4i5xa6l\nlR0bESNIrmx8WTr02551AkYAd0XEcGADsMNl/XeFg6AdS8fJHwJmRMTvil1Pa0p3necAY4pdSwEd\nA4xNx8xnAl+UdF9xSyq8iFiR/vse8DDJlY7bsyqgKmfv9kGSYGgxDoJ2Kj1w+kugIiJuK3Y9rUFS\nH0k90+d7AV8GlhS3qsKJiO9ERP+IKCH5Vf7TEfGVIpdVUJL2Tk9+IB0eOQlo12cDRsS7wNuSDkmb\nTgRa9KSPNnH10V0lqQw4HugtqQr4x4j4ZXGrKrhjgPOB19Mxc4DrI2J2EWsqtM8A96Y3ReoAzIqI\nTJxSmSH7Aw+nty3pBNwfEf9Z3JJaxRXAjPSMoeXARS3ZeSZOHzUzs/p5aMjMLOMcBGZmGecgMDPL\nOAeBmVnGOQjMzDLOQWCWkpUsypAAAAHASURBVLQ1vaLltkeL/XpTUkmWrn5rbUsmfkdg1kib0stT\nmGWK9wjMdiK9/v0/p9fAf1nSZ9P2EklPS1og6SlJB6bt+0t6OL0vwnxJR6dddZT0i/ReCb9Pf/2M\npCvT+0YskDSzSJtpGeYgMPvUXnWGhs7NmbcuIo4AfkZyxU+AO4F7I+JIYAZwR9p+B/BMRAwluSbM\norT9YGBaRBwGvA+clbZPAYan/fxDoTbOrD7+ZbFZStL6iOiWp70S+GJELE8v5PduRPSS9FeSm/9s\nSdtXRkRvSdVA/4j4OKePEpLLYh+cTn8b6BwRP5T0nyQ3TnoEeCTnngpmrcJ7BGaNE/U8b4qPc55v\n5dNjdKcC00j2Hl6R5GN31qocBGaNc27Ov39On79ActVPgEnAc+nzp4BLofZGOT3q61RSB2BARMwB\nvg30AHbYKzErJH/zMPvUXjlXagX4z4jYdgrp30haQPKtfmLadgXJXaOuI7mD1LYrQl4FTJf0NZJv\n/pcCK+t5zY7AfWlYCLijpW9DaLYzPkZgthNZvFm6ZYuHhszMMs57BGZmGec9AjOzjHMQmJllnIPA\nzCzjHARmZhnnIDAzy7j/D+q5Kts4KtXVAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de3wV9Z3/8deHi8Rwl4sXbmFbKsaV\nAKYoiEVK6w+tBS9URayr1VKpV3r5Sat11a2tu7V4W9aKotU2BakulXbxDlZdFAkIqEEFIUqAQkQE\nuRP47B8zCSfh5EKSOYeceT8fjzxy5nJmPnOSx7zPfGfmO+buiIhIfDVLdwEiIpJeCgIRkZhTEIiI\nxJyCQEQk5hQEIiIxpyAQEYk5BYEcMjNrbmbbzKxnY86bTmb2ZTOL5Frqqss2sxfMbFwUdZjZL8zs\nd/V9/+HGzF43s8vD11eZ2SvprSgzKQhiINwRl//sN7OdCcNJd0g1cfd97t7G3T9pzHkPV2b2kpnd\nmmT8BWa21syaH8ry3P1Mdy9ohLq+YWbFVZb9b+5+dUOXnWRdV5nZvvB/ZquZvW1mZzX2eiQ9FAQx\nEO6I27h7G+AT4NsJ4w7aIZlZi9RXeVh7HPhukvHfBf7o7vtSXE+6vBb+D3UAHgFmmlnbNNckjUBB\nIJjZL83sSTObbmZfAJea2WAze9PMPjez9WZ2v5m1DOdvYWZuZjnh8B/D6c+a2Rdm9oaZ9T7UecPp\nZ5nZh2a2xcweMLP/LW8aSFJ3XWr8gZmtNLPNZnZ/wnubm9k9ZrbJzFYBI2v4iP4bOMbMhiS8vxNw\nNvBEODzKzJaE35Y/MbNf1PB5JzZ31FhH+E18efhZfWRmV4Xj2wN/BXomHN11Df+Wv094/3lm9l74\nGc01s+MTppWY2Y/M7J3w855uZq1q+BwAcPf9wB+ANsCXE5Z3WsLfY4mZfS3x8zKz34d/p81m9nTC\n+DlmVhqO/6uZdautBmlcCgIpdx7wJ6A98CRQBtwAdAZOI9hB/aCG918C/AI4iuCo498OdV4z6wrM\nBH4arnc1MKiG5dSlxrOBk4EBBAH3jXD8BOBMIA/4KnBhdStx9+3AU8BlCaMvBpa5+3vh8DZgHMG3\n5W8DN5jZOTXUXq62OjYA3wLaAd8HHjCzfu6+JVzPJwlHdxsT32hmJxDssK8DugAvAbPLwzJ0IfBN\n4J8IPqdkRz6VhEeMVwB7gDXhuB7AbOBfCf6uk4D/DgMTgv+tI4BcoCtwXzi+GfAw0BPoBexNmCYp\noiCQcq+7+1/dfb+773T3he6+wN3L3H0VMBUYVsP7n3L3QnffCxQA/esx7znAEnd/Jpx2D/BpdQup\nY42/dvct7l4MvJKwrguBe9y9xN03AXfVUC8EzUMXJnxjviwcV17LXHd/L/z8lgIzktSSTI11hH+T\nVR6YC7wMnF6H5UIQVrPD2vaGy24PnJIwz73u/o9w3X+j5r/bUDP7HNgJ/Bq4xN3L/z6Xhet6PvwM\nngOWAiPDkBgBTHD3ze6+191fDbev1N1nhf9zW4FfUbfPTRqRgkDKrUkcMLO+ZvY/ZvYPM9sK3EHw\nzbs6/0h4vYOg2eBQ5z0usQ4PekQsqW4hdayxTusCPq6hXoC/A1uBb5vZVwiOMKYn1DLYzF4Jmzi2\nAFclqSWZGusws3PMbIGZfRbuhM+s43LLl12xvLBJpwRIbHo5lL/b6+7egeAb/xxgaMK0XsDYsFno\n87DWU8MaegCfhkcxlZhZGzN7JGxO2wrMPYTtk0aiIJByVS9ZfAh4F/iyu7cDbgUs4hrWA93LB8zM\nqLzTqqohNa4n2EGVq/Hy1jCUniD45vtdYE7Ct2EIjgCeBnq4e3uCk6l1qaXaOszsSIImqV8DR4c7\n4RcSllvbZabrCHbQ5ctrRvD5rq1DXdVy9y8ImrSuNLN+4eg1wGPu3iHhp7W7/yac1tnM2iVZ3E+B\n3sCg8G/49YbUJvWjIJDqtAW2ANvDtuaazg80lr8BA83s22E79A0EbdtR1DgTuNHMuoXt2DfV4T1P\nEJyH+B4JzUIJtXzm7rvM7FSCZpmG1tGKoF29FNgXnnMYkTB9A8EOtrord2YCo8zsjPC8wE+BL4AF\ndaytWu5eCjxKcK4HgnMR55nZN8MT4FlmNtzMjnP3NQTnJ6aYWQcza5lwIrktwZHI5nD7D7pMV6Kn\nIJDq/Bj4F4Idx0MEJ5Aj5e4bgIuAycAm4EvA28DuCGp8kKC9/R1gIcE379rqWwm8RbCD/p8qkycA\nv7bgqqufE+yEG1SHu38OTARmAZ8BYwjCsnz6uwRHIcVhc0zXKvW+R/D5PEgQJiOBUeH5gsZwD0HQ\n5IbnYM4jCIZSgosAfsyBfcyl4e8PCQLsunB4MsF5i03AfODZRqpNDoHpwTRyuLLgRq11wBh3fy3d\n9YhkKh0RyGHFzEaGzQetCL5d7iX4Fi4iEVEQyOFmKLCKoHnh/wHnuXt1TUMi0gjUNCQiEnM6IhAR\nibkm17lY586dPScnJ91liIg0KYsWLfrU3ZNejt3kgiAnJ4fCwsJ0lyEi0qSYWbV3z6tpSEQk5hQE\nIiIxpyAQEYk5BYGISMwpCEREYk5BICIScwoCEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJOQWBiEjM\nKQhERGJOQSAiEnMKAhGRmIssCMzsUTPbaGbvVjPdzOx+M1tpZsvMbGBUtYiISPWiPCL4PTCyhuln\nAX3Cn/HAgxHWIiIZoqAAcnKgWbPgd0FBuiuKXtTbHNkTytz9VTPLqWGW0cAT7u7Am2bWwcyOdff1\nUdUkIk1bQQGMHw87dgTDH38cDAOMG5e+uqKUim1O5zmCbsCahOGScNxBzGy8mRWaWWFpaWlKihOR\nw8/NNx/YIZbbsSMYn6lSsc1N4mSxu09193x3z+/SJemzl0ViKW7NJJ98cmjjM0EqtjmdQbAW6JEw\n3D0cJ1IvcdspljcZfPwxuB9oMsjk7e7Z89DGZ4JUbHM6g2A2cFl49dCpwBadH5D6iuNOMY7NJHfe\nCdnZlcdlZwfjM1UqtjnKy0enA28Ax5tZiZldaWZXm9nV4SxzgFXASuBh4IdR1SKZL447xTg2k4wb\nB1OnQq9eYBb8njo1c08UQ2q22YKLdpqO/Px8LywsTHcZcphp1iw4EqjKDPbvT309qZCTExz5VNWr\nFxQXp7oaOdyZ2SJ3z082rUmcLBapTRzbjuPYTCLRUBBIRojjTjGOzSQSjchuKBNJpfKd3803B23k\nPXsGIZDpO8Vx4zJ/GyV6CgLJGNopitSPmoZERGJOQSAiEnMKAhGRmFMQiIjEnIJARCTmFAQZKm4d\nsIlI/eny0QwUx4d3iEj96YggA8WxAzYRqT8FQQaKY6+UIlJ/CoIMFMcO2ESk/hQEGSiOHbCJSP0p\nCDKQeqUUkUOhq4Yy0I4dMGAA/Nd/Ba9zcoIf9yAYREQSKQiaoN27gxO/q1cHT6Kq+nvDhuTva9Mm\nCITevYOf8tflv9u3T9UWiMjhREFwGCorg5KSyjv4xNfr1lV+LGOLFkHzT04OfPvblXf2Rx4Z3EdQ\ndRnz5sG2bZXX27HjweGQ+Lt165RsvoikmIIgDfbvD3bmyb7Nr14Na9bAvn0H5m/WDLp3D3bG3/hG\n5R10795w3HHQvHn16+vf/+Bx7vDZZ8nXX1QEc+bArl2V39OlS/KQ6N07uCIpK6tBH4uIpEksgqCg\nILVPrnKHjRuTf5svLg6+oe/ZU/k9xx4b7FCHDDm46aZ7dzjiiMat0Qw6dQp+8pM8zto9aGKqGhKr\nV8PixTBrFuzdW/k9xx1XfdNTjx7QsmXjboOINA7zxDaGJiA/P98LCwvrPH/V7hYguJSyIVfRuMPm\nzcm/TZe/3rmz8ns6d66+bb5Xr6b3bXrfPli/PnnQlR/V7N9/YP5mzYIwqK7pqbajGhFpGDNb5O5J\nvvbFIAhycoJv4FX16hXstKrzxRfV7+SKi2Hr1srzd+hQ/bfhnJzgRG2c7N1b+3mORC1bBkdrVZuc\nyl8ffbSueBJpiJqCIOObhqrrVuHjj2H58up3VJ99Vnn+1q0P7JiGDTt4Z9WhQ6Sb0eS0bHng80lm\n167gb5Pss3/mGSgtrTx/VlbNJ7I7dVJQiNRXxgdBz57JjwgAcnMPvG7V6sCO5atfPXhH07mzdjSN\nKSsLvvKV4CeZ7duDYEjW9Pbmm0HTXKI2bRQKkvm++10YPrzxl5vxQXDnnQefI2jeHM4/H849t3LT\nQzPdZ33YaN0aTjwx+Elmy5bkRxOrV8OSJamsVCR1vv71aJab8UFQfkI4lVcNSfTat4e8vOBHRBom\n44MAgp2+dvwiIsmpMUREJOYUBCIiMacgEBGJuUiDwMxGmtkHZrbSzCYlmd7TzOaZ2dtmtszMzo6y\nHhEROVhkQWBmzYEpwFlALjDWzHKrzHYLMNPdBwAXA/8VVT0iIpJclEcEg4CV7r7K3fcAM4DRVeZx\noF34uj1QpeMBERGJWpRB0A1YkzBcEo5LdBtwqZmVAHOA65ItyMzGm1mhmRWWVu17QEREGiTdJ4vH\nAr939+7A2cAfzOygmtx9qrvnu3t+ly5dUl6kiEgmizII1gI9Eoa7h+MSXQnMBHD3N4AsoHOENYmI\nSBVRBsFCoI+Z9TazIwhOBs+uMs8nwAgAMzuBIAjU9iMikkKRBYG7lwHXAs8DywmuDnrPzO4ws1Hh\nbD8Gvm9mS4HpwOXe1B6QICLSxEXa15C7zyE4CZw47taE10XAaVHWICIiNUv3yWIREUkzBYGISMwp\nCEREYk5BICIScwoCEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJOQWBiEjMKQhERGJOQSAiEnMKAhGR\nmFMQiIjEnIJARCTmFAQiIjGnIBARiTkFgYhIzCkIRERiTkEgIhJzCgIRkZhTEIiIxJyCQEQk5hQE\nIiIxpyAQEYk5BYGISMwpCEREYk5BICIScwoCEZGYUxCIiMRcpEFgZiPN7AMzW2lmk6qZ50IzKzKz\n98zsT1HWIyIiB2tR00Qz+1FN0919cg3vbQ5MAb4JlAALzWy2uxclzNMH+BlwmrtvNrOuh1K8iIg0\nXI1BALRtwLIHASvdfRWAmc0ARgNFCfN8H5ji7psB3H1jA9YnIiL1UGMQuPvtDVh2N2BNwnAJcEqV\neb4CYGb/CzQHbnP356ouyMzGA+MBevbs2YCSRESkqtqahu6vabq7X98I6+8DnAF0B141s5Pc/fMq\n65kKTAXIz8/3Bq5TREQS1NY0tKgBy14L9EgY7h6OS1QCLHD3vcBqM/uQIBgWNmC9IiJyCGprGnq8\nActeCPQxs94EAXAxcEmVef4CjAUeM7POBE1FqxqwThEROUS1HREAYGZdgJuAXCCrfLy7f72697h7\nmZldCzxP0P7/qLu/Z2Z3AIXuPjucdqaZFQH7gJ+6+6Z6b42IiByyOgUBUAA8CXwLuBr4F6C0tje5\n+xxgTpVxtya8duBH4Y+IiKRBXW8o6+Tu04C97v53d/8eUO3RgIiINB11PSLYG/5eb2bfAtYBR0VT\nkoiIpFJdg+CXZtYe+DHwANAOmBhZVSIikjJ1CgJ3/1v4cgswPLpyREQk1ep0jsDMHjezDgnDHc3s\n0ejKEhGRVKnryeJ+iXf7hn0DDYimJBERSaW6BkEzM+tYPmBmR1H38wsiInIYq+vO/LfAG2b253D4\nO8Cd0ZQkIiKpVNeTxU+YWSEH7h04P/G5AiIi0nQdyhPKjgK2u/t/AqVhH0IiItLE1fWqoX8l6Gvo\nZ+GolsAfoypKRERSp65HBOcBo4DtAO6+joY9vUxERA4TdQ2CPWEHcQ5gZq2jK0lERFKprkEw08we\nAjqY2feBl4BHoitLRERSpa5XDd1tZt8EtgLHA7e6+4uRViYiIilR55vCwh3/iwBm1szMxrl7QWSV\niYhIStTYNGRm7czsZ2b2n2Z2pgWuJXic5IWpKVFERKJU2xHBH4DNwBvAVcDPAQPOdfclEdcmIiIp\nUFsQ/JO7nwRgZo8A64Ge7r4r8spERCQlartqqPzJZLj7PqBEISAikllqOyLIM7Ot4WsDjgyHjeDZ\n8+0irU5ERCJXYxC4e/NUFSIiIulxKJ3OiYhIBlIQiIjEnIJARCTmFAQiIjGnIBARiTkFgYhIzCkI\nRERiTkEgIhJzCgIRkZiLNAjMbKSZfWBmK81sUg3zXWBmbmb5UdYjIiIHiywIzKw5MAU4C8gFxppZ\nbpL52gI3AAuiqkVERKoX5RHBIGClu69y9z3ADGB0kvn+Dfh3QL2aioikQZRB0A1YkzBcEo6rYGYD\ngR7u/j81LcjMxptZoZkVlpaWNn6lIiIxlraTxWbWDJgM/Li2ed19qrvnu3t+ly5doi9ORCRGogyC\ntUCPhOHu4bhybYF/Bl4xs2LgVGC2ThiLiKRWlEGwEOhjZr3N7AjgYmB2+UR33+Lund09x91zgDeB\nUe5eGGFNIiJSRWRB4O5lwLXA88ByYKa7v2dmd5jZqKjWKyIih6a2R1U2iLvPAeZUGXdrNfOeEWUt\nIiKSnO4sFhGJOQWBiEjMKQhERGJOQSAiEnMKAhGRmFMQiIjEnIJARCTmFAQiIjGnIBARiTkFgYhI\nzCkIRERiTkEgIhJzCgIRkZhTEIiIxJyCQEQk5hQEIiIxpyAQEYk5BYGISMwpCEREYk5BICIScwoC\nEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJOQWBiEjMKQhERGJOQSAiEnMKAhGRmFMQiIjEXKRBYGYj\nzewDM1tpZpOSTP+RmRWZ2TIze9nMekVZj4iIHCyyIDCz5sAU4CwgFxhrZrlVZnsbyHf3fsBTwH9E\nVY+IiCQX5RHBIGClu69y9z3ADGB04gzuPs/dd4SDbwLdI6xHRESSiDIIugFrEoZLwnHVuRJ4NtkE\nMxtvZoVmVlhaWtqIJYqIyGFxstjMLgXygd8km+7uU909393zu3TpktriREQyXIsIl70W6JEw3D0c\nV4mZfQO4GRjm7rvrs6K9e/dSUlLCrl276lWoHL6ysrLo3r07LVu2THcpIhkryiBYCPQxs94EAXAx\ncEniDGY2AHgIGOnuG+u7opKSEtq2bUtOTg5m1pCa5TDi7mzatImSkhJ69+6d7nJEMlZkTUPuXgZc\nCzwPLAdmuvt7ZnaHmY0KZ/sN0Ab4s5ktMbPZ9VnXrl276NSpk0Igw5gZnTp10pGeSMSiPCLA3ecA\nc6qMuzXh9Tcaa10Kgcykv6tI9A6Lk8UiIpI+sQyCggLIyYFmzYLfBQUNW96mTZvo378//fv355hj\njqFbt24Vw3v27KnTMq644go++OCDGueZMmUKBQ0tNjR06FCOP/548vLyGDRoEMuWLWuU5ZZbuXIl\n/fv3B+Cll17i3HPPbdTli0jjibRp6HBUUADjx8OO8Da2jz8OhgHGjavfMjt16sSSJUsAuO2222jT\npg0/+clPKs3j7rg7zZolz97HHnus1vVcc8019SuwGk8++ST9+/fn4Ycf5qabbuLZZ5PexiEiGS52\nRwQ333wgBMrt2BGMb2wrV64kNzeXcePGceKJJ7J+/XrGjx9Pfn4+J554InfccUfFvEOHDmXJkiWU\nlZXRoUMHJk2aRF5eHoMHD2bjxuCCqltuuYV77723Yv5JkyYxaNAgjj/+eObPnw/A9u3bueCCC8jN\nzWXMmDHk5+dXhFR1Bg8ezNq1B67sffbZZxk8eDADBw7koosuYvv27QAsWLCAwYMHk5eXxymnnMKO\nHTv46KOPOP300xkwYAAnn3wyCxYsaNTPUESiF7sg+OSTQxvfUO+//z4TJ06kqKiIbt26cdddd1FY\nWMjSpUt58cUXKSoqOug9W7ZsYdiwYSxdupTBgwfz6KOPJl22u/PWW2/xm9/8piJUHnjgAY455hiK\nior4xS9+wdtvv11rjc8991xF083GjRu56667ePnll1m8eDH9+vXjvvvuY9euXVx88cVMmTKFpUuX\n8sILL9CqVSuOPfZYXnzxRd5++20KCgq4/vrrG/BpiUg6xK5pqGfPoDko2fgofOlLXyI/P79iePr0\n6UybNo2ysjLWrVtHUVERubmV++I78sgjOeusswA4+eSTee2115Iu+/zzz6+Yp7i4GIDXX3+dm266\nCYC8vDxOPPHEamu76KKL2L17Nzt37qw4apg/fz5FRUUMGTIEgD179jB06FCWL19Oz549GThwIADt\n27cHYPfu3Vx77bUsXbqUFi1a8NFHHx3S5yMi6Re7I4I774Ts7MrjsrOD8VFo3bp1xesVK1Zw3333\nMXfuXJYtW8bIkSOTXiN/xBFHVLxu3rw5ZWVlSZfdqlWrWuepyZNPPsnq1asZO3YsN9xwAxAcZYwc\nOZIlS5awZMkSioqKmDp1arXL+O1vf0uPHj145513eOutt9i9u143h4tIGsUuCMaNg6lToVcvMAt+\nT51a/xPFh2Lr1q20bduWdu3asX79ep5//vlGX8dpp53GzJkzAXjnnXeSNj0lMjN+9atf8eqrr7Ji\nxQqGDBnC3//+d1atWgUE5xxWrFhBbm4un3zyCYsXL67Yln379rFlyxaOPfZYzIzHH38cd2/0bRKR\naMUuCCDY6RcXw/79we9UhADAwIEDyc3NpW/fvlx22WWcdtppjb6O6667jrVr15Kbm8vtt99Obm5u\nRTNOdbKzs5k4cSJ33303Rx99NNOmTeOiiy4iLy+PIUOG8OGHH9KqVSumT5/OhAkTyMvL48wzz6xo\nFnrkkUfIy8tj9erVFUcpItJ0WFP7Bpefn++FhYWVxi1fvpwTTjghTRUdXsrKyigrKyMrK4sVK1Zw\n5plnsmLFClq0aLqng/T3FWk4M1vk7vnJpjXdvYMktW3bNkaMGEFZWRnuzkMPPdSkQ0BEoqc9RIbp\n0KEDixYtSncZItKExPIcgYiIHKAgEBGJOQWBiEjMKQhERGJOQdAIhg8fftDNYffeey8TJkyo8X1t\n2rQBYN26dYwZMybpPGeccQZVL5et6t5772VHQk96Z599Np9//nldSq/RbbfdVtGldm5uLtOnT2/w\nMqvKycnh008/BQ58HiKSWgqCRjB27FhmzJhRadyMGTMYO3Zsnd5/3HHH8dRTT9V7/VWDYM6cOXTo\n0KHey0s0ceJElixZwjPPPMMPfvAD9u7d2yjLFZHDR8ZdPnrjjVBLr8uHrH9/CHt/TmrMmDHccsst\n7NmzhyOOOILi4mLWrVvH6aefzrZt2xg9ejSbN29m7969/PKXv2T06NGV3l9cXMw555zDu+++y86d\nO7niiitYunQpffv2ZefOnRXzTZgwgYULF7Jz507GjBnD7bffzv3338+6desYPnw4nTt3Zt68eeTk\n5FBYWEjnzp2ZPHlyRe+lV111FTfeeCPFxcWcddZZDB06lPnz59OtWzeeeeYZjjzyyGq3sU+fPmRn\nZ7N582a6du3KRx99xDXXXENpaSnZ2dk8/PDD9O3blw0bNnD11VdXdFHx4IMPMmTIEM4991zWrFnD\nrl27uOGGGxhf/hAIEUm7jAuCdDjqqKMYNGgQzz77LKNHj2bGjBlceOGFmBlZWVnMmjWLdu3a8emn\nn3LqqacyatSoap/F++CDD5Kdnc3y5ctZtmxZRW+fAHfeeSdHHXUU+/btY8SIESxbtozrr7+eyZMn\nM2/ePDp37lxpWYsWLeKxxx5jwYIFuDunnHIKw4YNo2PHjqxYsYLp06fz8MMPc+GFF/L0009z6aWX\nVruNixcvpk+fPnTt2hWA8ePH87vf/Y4+ffqwYMECfvjDHzJ37lyuv/56hg0bxqxZs9i3bx/btm0D\n4NFHH+Woo45i586dfPWrX+WCCy6gU6dODf3oRaQRZFwQ1PTNPUrlzUPlQTBt2jQg6M3z5z//Oa++\n+irNmjVj7dq1bNiwgWOOOSbpcl599dWKPv379etHv379KqbNnDmTqVOnUlZWxvr16ykqKqo0varX\nX3+d8847r6IH1PPPP5/XXnuNUaNG0bt374pHSSZ2Y13VPffcw2OPPcaHH37IX//6VyC4e3n+/Pl8\n5zvfqZivvNfRuXPn8sQTTwBBr6jl/Rzdf//9zJo1C4A1a9awYsUKBYHIYSLjgiBdRo8ezcSJE1m8\neDE7duzg5JNPBqCgoIDS0lIWLVpEy5YtycnJSdr1dG1Wr17N3XffzcKFC+nYsSOXX355vZZTLrFz\nuObNm1dqgko0ceJEfvKTnzB79myuvPJKPvroI/bv30+HDh1qffJZuVdeeYWXXnqJN954g+zsbM44\n44wG1S4ijUsnixtJmzZtGD58ON/73vcqnSTesmULXbt2pWXLlsybN4+Pkz0VJ8HXvvY1/vSnPwHw\n7rvvVjxUfuvWrbRu3Zr27duzYcOGSs8Xbtu2LV988cVByzr99NP5y1/+wo4dO9i+fTuzZs3i9NNP\nr9f2jRo1ivz8fB5//HHatWtH7969+fOf/wwERz1Lly4FYMSIETz44IMAFd1Ub9myhY4dO5Kdnc37\n77/Pm2++Wa8aRCQaCoJGNHbsWJYuXVopCMaNG0dhYSEnnXQSTzzxBH379q1xGRMmTGDbtm2ccMIJ\n3HrrrRVHFnl5eQwYMIC+fftyySWXVOrCevz48YwcOZLhw4dXWtbAgQO5/PLLGTRoEKeccgpXXXUV\nAwYMqPf23XrrrUyePJn9+/dTUFDAtGnTKp6C9swzzwBw3333MW/ePE466SROPvlkioqKGDlyJGVl\nZZxwwglMmjSJU089td41iEjjUzfUctjT31ek4WrqhlpHBCIiMacgEBGJuYwJgqbWxCV1o7+rSPQy\nIgiysrLYtGmTdhoZxt3ZtGkTWVlZ6S5FJKNlxH0E3bt3p6SkhNLS0nSXIo0sKyuL7t27p7sMkYyW\nEUHQsmVLevfune4yRESapEibhsxspJl9YGYrzWxSkumtzOzJcPoCM8uJsh4RETlYZEFgZs2BKcBZ\nQC4w1sxyq8x2JbDZ3b8M3AP8e1T1iIhIclEeEQwCVrr7KnffA8wARleZZzTwePj6KWCEVdctp4iI\nRCLKcwTdgDUJwyXAKdXN4+5lZrYF6AR8mjiTmY0Hyjuw32ZmH9Szps5Vlx0D2uZ40DbHQ0O2uVd1\nE5rEyWJ3nwpMbehyzKywuji8fLQAAATfSURBVFusM5W2OR60zfEQ1TZH2TS0FuiRMNw9HJd0HjNr\nAbQHNkVYk4iIVBFlECwE+phZbzM7ArgYmF1lntnAv4SvxwBzXXeFiYikVGRNQ2Gb/7XA80Bz4FF3\nf8/M7gAK3X02MA34g5mtBD4jCIsoNbh5qQnSNseDtjkeItnmJtcNtYiINK6M6GtIRETqT0EgIhJz\nsQgCM3vUzDaa2bvpriVVzKyHmc0zsyIze8/Mbkh3TVEzsywze8vMlobbfHu6a0oFM2tuZm+b2d/S\nXUsqmFmxmb1jZkvMrLD2dzR9ZtbBzJ4ys/fNbLmZDW7U5cfhHIGZfQ3YBjzh7v+c7npSwcyOBY51\n98Vm1hZYBJzr7kVpLi0y4V3prd19m5m1BF4HbnD3N9NcWqTM7EdAPtDO3c9Jdz1RM7NiIN/dY3Mz\nmZk9Drzm7o+EV2Fmu/vnjbX8WBwRuPurBFclxYa7r3f3xeHrL4DlBHdyZywPbAsHW4Y/Gf1Nx8y6\nA98CHkl3LRINM2sPfI3gKkvcfU9jhgDEJAjiLuzVdQCwIL2VRC9sJlkCbARedPdM3+Z7gf8P7E93\nISnkwAtmtijsfibT9QZKgcfCJsBHzKx1Y65AQZDhzKwN8DRwo7tvTXc9UXP3fe7en+BO9kFmlrFN\ngWZ2DrDR3Relu5YUG+ruAwl6Nr4mbPrNZC2AgcCD7j4A2A4c1K1/QygIMljYTv40UODu/53uelIp\nPHSeB4xMdy0ROg0YFbaZzwC+bmZ/TG9J0XP3teHvjcAsgp6OM1kJUJJwdPsUQTA0GgVBhgpPnE4D\nlrv75HTXkwpm1sXMOoSvjwS+Cbyf3qqi4+4/c/fu7p5DcFf+XHe/NM1lRcrMWocXPxA2j5wJZPTV\ngO7+D2CNmR0fjhoBNOpFH02i99GGMrPpwBlAZzMrAf7V3aelt6rInQZ8F3gnbDMH+Lm7z0ljTVE7\nFng8fChSM2Cmu8fiksoYORqYFT62pAXwJ3d/Lr0lpcR1QEF4xdAq4IrGXHgsLh8VEZHqqWlIRCTm\nFAQiIjGnIBARiTkFgYhIzCkIRERiTkEgEjKzfWGPluU/jXb3ppnlxKn3W2laYnEfgUgd7Qy7pxCJ\nFR0RiNQi7P/+P8I+8N8ysy+H43PMbK6ZLTOzl82sZzj+aDObFT4XYamZDQkX1dzMHg6flfBCePcz\nZnZ9+NyIZWY2I02bKTGmIBA54MgqTUMXJUzb4u4nAf9J0OMnwAPA4+7eDygA7g/H3w/83d3zCPqE\neS8c3weY4u4nAp8DF4TjJwEDwuVcHdXGiVRHdxaLhMxsm7u3STK+GPi6u68KO/L7h7t3MrNPCR7+\nszccv97dO5tZKdDd3XcnLCOHoFvsPuHwTUBLd/+lmT1H8OCkvwB/SXimgkhK6IhApG68mteHYnfC\n630cOEf3LWAKwdHDQjPTuTtJKQWBSN1clPD7jfD1fIJePwHGAa+Fr18GJkDFg3LaV7dQM2sG9HD3\necBNQHvgoKMSkSjpm4fIAUcm9NQK8Jy7l19C2tHMlhF8qx8bjruO4KlRPyV4glR5j5A3AFPN7EqC\nb/4TgPXVrLM58McwLAy4v7EfQyhSG50jEKlFHB+WLvGipiERkZjTEYGISMzpiEBEJOYUBCIiMacg\nEBGJOQWBiEjMKQhERGLu/wDwcMp93xox0QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU9fX/8ddh0bAKChZlCyr7jgG1\nooILYlXcF4pa3BBaF+wmX/261G/92cWq1SqIW7WiSFUUFavVaoWqSEBAQVmKUQOogAKyCQnn98e9\nSSbJJBlCZiaZ+34+Hnlk5t47d84dwn3P53Pv/Vxzd0REJLrqpbsAERFJLwWBiEjEKQhERCJOQSAi\nEnEKAhGRiFMQiIhEnIJASjGz+ma22cw61OSy6WRmh5hZUs6TLrtuM3vNzEYlow4zu9HMJlX39elm\nZj8xs1cSWO4hM7s+FTVJwHQdQd1mZptjnjYGvgcKw+dXuPuU1FdVu5jZIcByd7c4814H3nb3W8tM\nPwu4B+jg7oVlX5fIuvdw2eOBh9w9u6pl95SZXQY8AGwDdgH/BW5w95nJfm+pHdQiqOPcvWnRD/A5\ncGrMtHIhYGYNUl9lrfYYcGGc6RcCT1QWAhlmVvg31BJ4HPi7me1TdiH9/WQmBUGGM7PfmtnTZvaU\nmX0HXGBmR5jZe2a2wczWmNk9ZtYwXL6BmbmZZYfPnwjnv2Jm35nZu2bWaXeXDeefZGbLzGyjmd1r\nZv8xs9EV1J1IjVeY2Qoz+9bM7ol5bX0zu8vM1pvZSmB4JR/Rc0AbM/thzOv3A35EsEPEzEaY2QIz\n22Rmn5vZjZV83rOLtqmqOszsMjP7OPys/ht+MyfcAb8IdAi73jab2f7hv+VfY15/hpktDj+jf5lZ\n15h5+Wb2czP7MPy8nzKzvSv5HAAIg+8RgtZlJzM73szyzOx6M/sSeDDmM1kYvvdsM+sV894dzex5\nM1trZuvM7M8x2/tW+Lhe+G/6dVjfIjPrEc57wsxuiVnf2PDfeX243gPC6ZX+HUjiFATRcAbwJLAP\n8DRQAFwDtAKOJNhBXVHJ638M3AjsS9Dq+L/dXdbM9gemAb8K3/dTYFAl60mkxh8BhwL9CQLu+HD6\nOGAY0BcYCJxb0Zu4+xbgGeCimMnnA4vcfXH4fDMwCmgBnApcY2anVFJ7karq+Ao4GWgOXA7ca2Z9\n3H1j+D6fx7Tuvo59oZl1B/4GXAW0Bl4HZhSFZehc4ATgIILPKV7LpxQLvvFfCnxH0EUE0A5oCnQA\nfmpmAwkC4TJgP4LgeMHM9gpf/zKwAsgG2hP8u5d1EnA40JmgFXI+8E2ceoYBtwJnA22B1UDZlm5F\nfweSIAVBNMx29xfdfZe7b3P3ue4+x90L3H0lMBk4ppLXP+Puue6+k+A/Yb9qLHsKsMDdXwjn3QWs\nq2glCdZ4u7tvdPc84K2Y9zoXuMvd8919PfC7SuqFoHvo3JhvzBeF04pq+Ze7Lw4/v4XA1Di1xFNp\nHeG/yUoP/At4AzgqgfVCsOOcEda2M1z3PsBhMcvc7e5fhu/9EpX/uw02sw3AlwQ73dPd/btwXgFw\ni7vvcPdtwBjg/vDfqNDdHwmXGwgcQRDe17n7lvDv7T9x3m8nQQB2Cz+LJe7+ZZzlRhEcK1ng7tuB\nCcAxZtYuZpmK/g4kQQqCaPgi9omZdTOzl83sSzPbRPCNq1Ulr4/9D7qV4Nvh7i57YGwdHpylkF/R\nShKsMaH3Aj6rpF6AfwObgFPNrAvBN8unYmo5wszeCrs6NhJ8E67s8ypSaR1mdoqZzTGzb8Kd8LAE\n11u07uL1ufsugs+zbcwyu/PvNtvdW7h7K3f/YRhMRb5y9x0xzzsC14XdQhvC2g8I37s9kFfVsRV3\nfw2YBEwEvjKzSWbWLIHt3AR8uwfbKXEoCKKh7KlhDwAfAYe4e3PgJqDKM1n20BqCLgYAzMwo/Z+5\nrD2pcQ3BDqlIpae3hqH0OEFL4EJgprvHtlamAs8C7d19H+ChBGupsA4za0TQJXU78AN3bwG8FrPe\nqk7nW02wQy5aXz2Cz3dVAnXtrrK1fAH8JgyOop/G7j4tnNfRzOpXuVL3u919ANAL6AH8PM5iZbez\nGUFXUjK2M7IUBNHUDNgIbAn7mis7PlBTXgIGmNmpYT/yNQR928mocRow3szahgd+r0vgNY8THIe4\nhJhuoZhavnH37WZ2OEG3zJ7WsTewF7AWKAyPORwXM/8roFUF35KL1j3CzIaExwV+RdCvPyfB2vbE\ng8DPzGygBZqG/65NgHeB9cD/M7PGZtbIzI4suwIzGxT+NAC2ADsITl0t6yngUjPrE3bd3U5whlOF\nrUnZfQqCaPoF8BOCHccDBAeQk8rdvwLOA+4k2FEcDHxAcN1DTdc4kaC//UNgLsE376rqWwG8T7CD\nfrnM7HHA7RacdXU98Q9+7lYd7r4BuBaYTnCQ9GyCsCya/xFBKyQv7H7Zv0y9iwk+n4kEYTIcGBEe\nL0gqd3+P4DOZSNBNswy4IJxXQHA8qDtB6+Bzgm0rqwXwMLAByCNoPd0Z573+QdAtOD1cpgPBcQOp\nQbqgTNIi7DpYDZzt7rPSXY9IlKlFICljZsPNrEXYxL+R4MyR99NclkjkKQgklQYDKwm6Mk4EznD3\nirqGRCRF1DUkIhJxahGIiERcnRtAqlWrVp6dnZ3uMkRE6pR58+atc/e4p2zXuSDIzs4mNzc33WWI\niNQpZlbhFfbqGhIRiTgFgYhIxCkIREQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYk4\nBYGISMQpCEREIk5BICIScQoCEZGIUxCIiEScgkBEJOKSFgRm9oiZfW1mH1Uw38zsHjNbYWaLzGxA\nsmoREZGKJbNF8FdgeCXzTwI6hz9jgIlJrEVEpM6aMgWys6FeveD3lCk1u/6kBYG7vw18U8kipwGP\ne+A9oIWZHZCsekQyUbJ3ELVR1LZ5yhQYMwY++wzcg99jxtTsdqfzGEFb4IuY5/nhtHLMbIyZ5ZpZ\n7tq1a1NSnEhtl4odRG0TxW2+4QbYurX0tK1bg+k1pU4cLHb3ye6e4+45rVvHvfeySOS+KaZiB1Hb\nRHGbP/9896ZXRzqDYBXQPuZ5u3CayG6L4jfFVOwgapsobnOHDrs3vTrSGQQzgIvCs4cOBza6+5o0\n1iN1WBS/KaZiB1HbRHGbb7sNGjcuPa1x42B6TUnm6aNPAe8CXc0s38wuNbOxZjY2XGQmsBJYATwI\n/DRZtUjmi+I3xVTsIGqbKG7zqFEweTJ07Ahmwe/Jk4PpNcXcvebWlgI5OTmem5ub7jKklsnODrqD\nyurYEfLyUl1N6kyZErR6Pv88+FZ82201u4OojaK4zTXBzOa5e07ceQoCyQRFxwhiu4caN675b04i\ndVVlQVAnzhoSqUoqms8imapBugsQqSmjRmnHL1IdahGIiEScgiBDRe3iKhGpPnUNZaCyB06LLq4C\ndZ2ISHlqEWSgKF5cJSLVpyDIQFG8uEpEqk9BkIGieBm+iFSfgiADRfEyfBGpPgVBBtLFVSKyO3TW\nUIbSxVUikii1CEREIi4yQbB5c/lTKkVEJEJB8OijsN9+cMopMGkS5OenuyIRkdohMkHwwx/CFVfA\nxx/DuHHQvj307w833QRz58KuXemuUEQkPSJ3PwL3IAxefBFeegneeScIgTZt4OST4dRT4fjjoUmT\nGixaRCTNdGOaSqxbB6+8EoTCP/4BmzbB3nvDsccGoXDKKUHrQUSkLov8jWkqG4mzVSu48EJ4+mlY\nuxZefz3oOlq2DH760+Bq3P794cYb4f331YUkIpkn41sE1b2FoTt88klJF9J//hOEwA9+UNKFdMIJ\n6kISkboh0l1DNXVT8/XrS3chbdwYdCENHVrShaSxfESktop0ENSrF3y7L8us+t08O3fCrFlBa+HF\nF+G//w2m9+kThMKpp8LAgcF7i4jUBpEOgppqEVTEHZYuLelCmj07CJj99y/dhdS06Z6/l4hIdUX6\nYHGyR+I0g27d4Fe/gn//Ozjg/MQTwVlHzz0HZ54ZXMg2fDjcd1/8UBIRSaeMbxFAcMD4hhuCG7N0\n6BCEQCoGZNu5M2ghvPRS0GJYvjyY3rt3SRfSoEHqQtoTW7bAp5+W/OTlQYMGMGAAHHooHHywPl8R\niHjXUG1StgupsDDoQvrRj0q6kJo1S3eVtcuOHUErqmgnH7vT//TToAUWq1Gj4HPdsSN43rx5SSgU\n/RxyiMJBokdBUAt9801w9tGLLwZnI23cCHvtBUOGlLQWOnZMd5XJV1gIq1bF38l/+mkwL/ZPtEGD\n4HPp1Kn8T3Z2EKw7d8LixTBvXsnPokXw/ffBOpo3D64NiQ2Hzp0VDpLZFAS13M6dwXUKRa2FZcuC\n6b17B6elFnUh1a+f3jqrwz341h5vJ//pp0F33c6dJcubQdu25XfwRY/btq3e5xAvHBYuLAmHZs3K\nh0OXLgqH2qTob2nZMlixArZtS3dFqXf00dCzZ/VeqyCoY5YtKwmFWbOCb82tW5d0IQ0bVru6kDZu\nrHhHn5dXfvjv1q0r3tF36BBcn5EKO3fCkiXlw2H79mB+06bxw6EuBnJdsmFDcDxt2bLyvzdtSnd1\n6TVxIowdW73XKgjqsG+/DbqQXnop6EL69lto2LB0F1J2dnJr2Lat4q6bvLygpljNm5ffwcfu9Gvz\nqbQFBSXhMH9+8HvBgpJvn02alA+Hrl0VDrtry5bgW328HX7scZ+iW6126RJ03xX9PuSQ4O8sapo1\nK38WZKIUBBmioKB0F9LSpcH0nj1LQuGww3Z/p7RzJ3zxRfkdfNHjL78svXxWVrBDj7ej79QJWrYM\n/gNnioKCYLiR2JbDggUlLZ0mTaBfv9Lh0K2bwmHHDli5Mv43+1WrSi974IGld/RFvw86KPh7kz2X\ntiAws+HAn4H6wEPu/rsy8zsAjwEtwmUmuPvMytYZ5SAoa/nyklB4++2gC6lVq9JdSM2bBxe4rVkT\nfyf/6adBCMReZV2/fjDiarydfKdOwXhLUe87LywsHw4ffFASDo0bxw+HBhl2l/DCwuCsrnjf7PPy\nSv9d7bdf+R19ly7Bt/va3ErMFGkJAjOrDywDTgDygbnASHdfErPMZOADd59oZj2Ame6eXdl6FQTx\nbdhQ0oU0c2ZJF1KHDsHd2IoOihY54ICKd/Tt2mXeDisVCguDVlrZcNiyJZjfqFH5cOjevfZ/1u6w\nenX8b/YrV5acqgtB10W8b/adO8O++6ZvGyR9QXAEcIu7nxg+/x8Ad789ZpkHgJXu/vtw+T+5+w8r\nW6+CoGoFBcENd158MTgrJ/Z0y+zs4HmjRumuMhoKC4MdZtlw2Lw5mN+oEfTtG4RC0fUOPXoEIZ5K\n7sG9OeJ9s1++vPQB/733Ltm5l93h/+AHmdUtmEnSFQRnA8Pd/bLw+YXAYe5+ZcwyBwCvAS2BJsDx\n7j4vzrrGAGMAOnTocOhnGqdB6rBdu+KHw3ffBfOzsoIBDGNbDj171kw4bNxYsnMvu8PfsKFkuQYN\ngv75st/qu3QJWoxR7xqsi2pzEPw8rOFPYYvgYaCXu1c4LqhaBJKJdu0Kdsax4TB/fkk47L13/HDY\na6/y69q2LTgjJ15XztdflyxnFnQdxuu379gx9a0SSa7KgiCZvZOrgNibPLYLp8W6FBgO4O7vmlkW\n0Ar4GpEIqVcvOA21a1f48Y+Dabt2BTv02HB48kmYNCmYv9deJeFgVrLD/+KL0us+4IBgBz9iROkd\n/sEH64wcCSQzCOYCnc2sE0EAnA/8uMwynwPHAX81s+5AFlBm9BiRaKpXL9hpd+kCI0cG03btCu5/\nEdtqePrpkmWHDCl/Rk5tuvhQaqekBYG7F5jZlcCrBKeGPuLui83sViDX3WcAvwAeNLNrAQdGe127\nsEEkherVK+mvP//8dFcjmSKpJ66F1wTMLDPtppjHS4Ajk1mDiIhUTsf+RUQiTkEgIhJxCgIRkYhT\nEIiIRJyCQEQk4hQEIiIRpyAQEYk4BYGISMQpCEREIk5BICIScQoCEZGIUxCIiEScgkBEJOIUBCIi\nEacgEBGJOAWBiEjEKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTiFAQiIhGnIBARibgGiS5oZm2B\njrGvcfe3k1GUiIikTkJBYGa/B84DlgCF4WQHFAQiInVcoi2C04Gu7v59MosREZHUS/QYwUqgYTIL\nERGR9Ei0RbAVWGBmbwDFrQJ3vzopVYmISMokGgQzwh8REckwCQWBuz9mZnsBXcJJS919Z/LKEhGR\nVEnoGIGZDQGWA/cB9wPLzOzoBF433MyWmtkKM5tQwTLnmtkSM1tsZk/uRu0iIlIDEu0a+hMwzN2X\nAphZF+Ap4NCKXmBm9QmC4wQgH5hrZjPcfUnMMp2B/wGOdPdvzWz/6m2GiIhUV6JnDTUsCgEAd19G\n1WcRDQJWuPtKd98BTAVOK7PM5cB97v5tuN6vE6xHRERqSKJBkGtmD5nZkPDnQSC3ite0Bb6IeZ4f\nTovVBehiZv8xs/fMbHi8FZnZGDPLNbPctWvXJliyiIgkItEgGEdwVfHV4c+ScNqeagB0BoYAI4EH\nzaxF2YXcfbK757h7TuvWrWvgbUVEpEiiZw19D9wZ/iRqFdA+5nm7cFqsfGBOeAbSp2a2jCAY5u7G\n+4iIyB6otEVgZtPC3x+a2aKyP1Wsey7Q2cw6haeenk/5axGeJ2gNYGatCLqKVlZjO0REpJqqahFc\nE/4+ZXdX7O4FZnYl8CpQH3jE3Reb2a1ArrvPCOcNM7Oiwex+5e7rd/e9RESk+szdq17IrAmwzd13\nhaeOdgNeScdFZTk5OZ6bW9VxahERiWVm89w9J968RA8Wvw1khfckeA24EPhrzZQnIiLplGgQmLtv\nBc4E7nf3c4CeyStLRERSJeEgMLMjgFHAy+G0+skpSUREUinRIBhPMBTE9PCA70HAm8krS0REUiXR\n6wj+Dfw75vlKggvLRESkjqs0CMzsbncfb2YvEtyjuBR3H5G0ykREJCWqahH8Lfx9R7ILERGR9Kg0\nCNx9Xvgwl/A6AigeYnrvJNcmIiIpkOjB4jeAxjHPGwGv13w5IiKSaokGQZa7by56Ej5uXMnyIiJS\nRyQaBFvMbEDREzM7FNiWnJJERCSVEr1V5Xjg72a2GjCgDXBe0qoSEZGUSfQ6grlm1g3oGk5amo4B\n50REpOYl1DVkZo2B64Br3P0jINvMdntoahERqX0SPUbwKLADOCJ8vgr4bVIqEhGRlEo0CA529z8A\nOwHCkUgtaVWJiEjKJBoEO8ysEeEwE2Z2MPB90qoSEZGUSfSsoZuBfwDtzWwKcCQwOllFiYhI6lQZ\nBGZmwCcEN6U5nKBL6Bp3X5fk2kREJAWqDAJ3dzOb6e69KbkpjYiIZIhEjxHMN7OBSa1ERETSItFj\nBIcBF5hZHrCFoHvI3b1PsgoTEZHUSDQITkxqFSIikjZV3aEsCxgLHAJ8CDzs7gWpKExERFKjqmME\njwE5BCFwEvCnpFckIiIpVVXXUI/wbCHM7GHg/eSXJCIiqVRVi6B4hFF1CYmIZKaqWgR9zWxT+NiA\nRuHzorOGmie1OhERSbqqbl5fP1WFiIhIeiR6QZmIiGQoBYGISMQlNQjMbLiZLTWzFWY2oZLlzjIz\nN7OcZNYjIiLlJS0IzKw+cB/B9Qc9gJFm1iPOcs2Aa4A5yapFREQqlswWwSBghbuvdPcdwFTgtDjL\n/R/we2B7EmsREZEKJDMI2gJfxDzPD6cVM7MBQHt3r3R4azMbY2a5Zpa7du3amq9URCTC0naw2Mzq\nAXcCv6hqWXef7O457p7TunXr5BcnIhIhyQyCVUD7mOftwmlFmgG9gLfC4a0PB2bogLGISGolMwjm\nAp3NrJOZ7QWcD8womunuG929lbtnu3s28B4wwt1zk1iTiIiUkbQgCMcmuhJ4FfgYmObui83sVjMb\nkaz3FRGR3ZPojWmqxd1nAjPLTLupgmWHJLMWERGJT1cWi4hEnIJARCTiFAQiIhGnIBARiTgFgYhI\nxCkIREQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYk4BYGISMQpCEREIk5BICIScQoC\nEZGIUxCIiEScgkBEJOIUBCIiEacgEBGJOAWBiEjEKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTi\nFAQiIhGnIBARiTgFgYhIxCkIREQiLqlBYGbDzWypma0wswlx5v/czJaY2SIze8PMOiazHhERKS9p\nQWBm9YH7gJOAHsBIM+tRZrEPgBx37wM8A/whWfWIiEh8yWwRDAJWuPtKd98BTAVOi13A3d90963h\n0/eAdkmsR0RE4khmELQFvoh5nh9Oq8ilwCvxZpjZGDPLNbPctWvX1mCJIiJSKw4Wm9kFQA7wx3jz\n3X2yu+e4e07r1q1TW5yISIZrkMR1rwLaxzxvF04rxcyOB24AjnH376vzRjt37iQ/P5/t27dXq1DJ\nTFlZWbRr146GDRumuxSRWi2ZQTAX6GxmnQgC4Hzgx7ELmFl/4AFguLt/Xd03ys/Pp1mzZmRnZ2Nm\ne1KzZAh3Z/369eTn59OpU6d0lyNSqyWta8jdC4ArgVeBj4Fp7r7YzG41sxHhYn8EmgJ/N7MFZjaj\nOu+1fft29ttvP4WAFDMz9ttvP7USRRKQzBYB7j4TmFlm2k0xj4+vqfdSCEhZ+psQSUytOFgsIiLp\nE8kgmDIFsrOhXr3g95Qpe7a+9evX069fP/r160ebNm1o27Zt8fMdO3YktI6LL76YpUuXVrrMfffd\nx5Q9LTY0ePBgunbtSt++fRk8eDDLly/f43UWFhZy1FFHVbrMiSeeyHfffbfH7yUiNcfcPd017Jac\nnBzPzc0tNe3jjz+me/fuCb1+yhQYMwa2bi2Z1rgxTJ4Mo0bteX233HILTZs25Ze//GWp6e6Ou1Ov\nXu3I3sGDB/OXv/yFfv36cf/99/P666/z3HPPlVqmoKCABg2S2nuYdLvztyGSycxsnrvnxJtXO/ZK\nKXTDDaVDAILnN9xQ8++1YsUKevTowahRo+jZsydr1qxhzJgx5OTk0LNnT2699dbiZQcPHsyCBQso\nKCigRYsWTJgwgb59+3LEEUfw9dfBCVX/+7//y9133128/IQJExg0aBBdu3blnXfeAWDLli2cddZZ\n9OjRg7PPPpucnBwWLFhQaZ1HH300K1asAKBdu3ZMmDCB/v37M336dJYvX86JJ57IoYceytFHH82y\nZcsA+PLLLznttNPo06cPffv2Zc6cOcW1A6xatYrBgwfTr18/evXqVVxfu3bt2LBhAwB/+MMf6NWr\nF7169eLee+8t/sx69erFpZdeSs+ePTnppJN0wFckySIXBJ9/vnvT99Qnn3zCtddey5IlS2jbti2/\n+93vyM3NZeHChfzzn/9kyZIl5V6zceNGjjnmGBYuXMgRRxzBI488Enfd7s7777/PH//4x+JQuffe\ne2nTpg1Llizhxhtv5IMPPqiyxhdffJHevXsXP99///354IMPOOeccxgzZgz3338/8+bN4/bbb+fK\nK68E4Gc/+xknnHACixYtYt68eeW+dT/xxBOceuqpLFiwgIULF9KnT59S8+fMmcOUKVOYO3cu7777\nLvfffz8ffvghAEuXLmX8+PEsXryYRo0a8fzzz1e5DSJSfXW73V8NHTrAZ5/Fn54MBx98MDk5Ja2x\np556iocffpiCggJWr17NkiVL6NGj9Fh8jRo14qSTTgLg0EMPZdasWXHXfeaZZxYvk5eXB8Ds2bO5\n7rrrAOjbty89e/assLbzzjuPRo0acdBBBxV/Iy+aDrBhwwbee+89zjrrrOJ5BQUFALz11ltMnToV\ngAYNGtC8efPieQADBw7kiiuuYPv27Zx++un07du31HvPnj2bs846i0aNGgFw+umnM2vWLIYNG8Yh\nhxxSHEyx2yYiyRG5ILjttvjHCG67LTnv16RJk+LHy5cv589//jPvv/8+LVq04IILLojb7bHXXnsV\nP65fv36pHWysvffeu8plKvP000/Tr1+/Cmt2d1q1alVh11Jlp2cee+yxvPXWW7z88stcdNFF/PrX\nv2ZUggdhirYLqr9tIpK4yHUNjRoVHBju2BHMgt81daC4Kps2baJZs2Y0b96cNWvW8Oqrr9b4exx5\n5JFMmzYNgA8//DBu11OiWrZsyQEHHMD06dMB2LVrFwsXLgRg6NChTJo0CQjOFtq0aVOp13722We0\nadOGMWPGcPHFF5frojrqqKOYPn0627ZtY/PmzbzwwgtVnnEkIskRuSCAYKeflwe7dgW/UxECAAMG\nDKBHjx5069aNiy66iCOPPLLG3+Oqq65i1apV9OjRg9/85jf06NGDffbZp9rrmzp1KpMmTSruZnrp\npZcA+Mtf/sKrr75K7969ycnJ4ZNPPin1ujfeeIO+ffvSv39/nnvuOa666qpS8wcNGsTIkSMZOHAg\nhx9+OOPGjSt1nEJEUidyp49muoKCAgoKCsjKymL58uUMGzaM5cuX1/nTQKtLfxsigcpOH43m3iGD\nbd68meOOO46CggLcnQceeCCyISAiidEeIsO0aNGCefPmpbsMEalDInmMQERESigIREQiTkEgIhJx\nCgIRkYhTENSAoUOHlrs47O6772bcuHGVvq5p06YArF69mrPPPjvuMkOGDKHs6bJl3X333WyNuVT6\nRz/6UfHAbnvilltuKR5Su1evXsyYUa0byJVz2WWXVXqh20033cTrr79eI+8lIlVTENSAkSNHFo+7\nU2Tq1KmMHDkyodcfeOCBPPPMM9V+/7JBMHPmzOJRQPfUtddey4IFC/j73//OJZdcwq5du0rNr87w\nDw899FC58ZVi3XrrrRx/fI3dvE5EqpBxQTB+PAwZUrM/48dX/p5nn302L7/8cvFNaPLy8li9ejVH\nHXVU8Xn9AwYMoHfv3rzwwgvlXp+Xl0evXr0A2LZtG+effz7du3fnjDPOYNu2bcXLjRs3rngI65tv\nvhmAe+65h9WrVzN06FCGDh0KQHZ2NuvWrQPgzjvvLB7quWgI67y8PLp3787ll19Oz549GTZsWKn3\niad79+40aNCAdevWMXr0aFJ5uOIAAApoSURBVMaOHcthhx3Gr3/9a7Zs2cIll1zCoEGD6N+/f/E2\nFhYW8stf/pJevXrRp0+f4oHtilo5hYWFjB49ml69etG7d2/uuusuAEaPHl0cjG+88Qb9+/end+/e\nXHLJJXz//ffF23jzzTcXf65lr2wWkcTpOoIasO+++zJo0CBeeeUVTjvtNKZOncq5556LmZGVlcX0\n6dNp3rw569at4/DDD2fEiBEVDtg2ceJEGjduzMcff8yiRYsYMGBA8bzbbruNfffdl8LCQo477jgW\nLVrE1VdfzZ133smbb75Jq1atSq1r3rx5PProo8yZMwd357DDDuOYY46hZcuWLF++nKeeeooHH3yQ\nc889l2effZYLLrigwm2cM2cO9erVo3Xr1gDk5+fzzjvvUL9+fa6//nqOPfZYHnnkETZs2MCgQYM4\n/vjjefzxx8nLy2PBggU0aNCAb775ptQ6FyxYwKpVq/joo48AynVnbd++ndGjR/PGG2/QpUsXLrro\nIiZOnMj4MJlbtWrF/Pnzuf/++7njjjt46KGHEvwXE5FYGRcE4ZfelCvqHioKgocffhgIRvC8/vrr\nefvtt6lXrx6rVq3iq6++ok2bNnHX8/bbb3P11VcD0KdPn1Lj+E+bNo3JkydTUFDAmjVrWLJkSblx\n/mPNnj2bM844o3g00TPPPJNZs2YxYsQIOnXqVDzyaGVDPd9111088cQTNGvWjKeffro4wM455xzq\n168PwGuvvcaMGTO44447gGAH/vnnn/P6668zduzY4iub991331LrPuigg1i5ciVXXXUVJ598MsOG\nDSs1f+nSpXTq1IkuXboA8JOf/IT77ruvOAhih+Eue3c1EUlcxgVBupx22mlce+21zJ8/n61bt3Lo\noYcCMGXKFNauXcu8efNo2LAh2dnZ1brj1qeffsodd9zB3LlzadmyJaNHj96jO3eVHeq5oq6ha6+9\nttxtN6H08NruzrPPPkvXrl13q4aWLVuycOFCXn31VSZNmsS0adMqvAlPPHs6DLeIBDLuGEG6NG3a\nlKFDh3LJJZeUOki8ceNG9t9/fxo2bMibb77JZ/HuihPj6KOP5sknnwTgo48+YtGiRUAwhHWTJk3Y\nZ599+Oqrr3jllVeKX9OsWbO4N4Q/6qijeP7559m6dStbtmxh+vTpSRnq+cQTT+Tee++laADDoiGn\nTzjhBB544IHinXTZrqF169axa9cuzjrrLH77298yf/78UvO7du1KXl5e8W00//a3v3HMMcfUeP0i\nUacWQQ0aOXIkZ5xxRqkziEaNGsWpp55aPFxzt27dKl3HuHHjuPjii+nevTvdu3cvblkUDencrVs3\n2rdvX2oI6zFjxjB8+HAOPPBA3nzzzeLpAwYMYPTo0QwaNAgITtvs379/jd/x68Ybb2T8+PH06dOH\nXbt20alTJ1566SUuu+wyli1bRp8+fWjYsCGXX3558a0uIbiv8cUXX1x8JtLtt99ear1ZWVk8+uij\nnHPOORQUFDBw4EDGjh1bo7WLiIahlgynvw2RQGXDUKtrSEQk4hQEIiIRlzFBUNe6uCT59DchkpiM\nCIKsrCzWr1+v//hSzN1Zv349WVlZ6S5FpNbLiLOG2rVrR35+PmvXrk13KVKLZGVl0a5du3SXIVLr\nZUQQNGzYkE6dOqW7DBGROimpXUNmNtzMlprZCjObEGf+3mb2dDh/jpllJ7MeEREpL2lBYGb1gfuA\nk4AewEgzKzv28KXAt+5+CHAX8Ptk1SMiIvEls0UwCFjh7ivdfQcwFTitzDKnAY+Fj58BjrOKhuUU\nEZGkSOYxgrbAFzHP84HDKlrG3QvMbCOwH7AudiEzGwOMCZ9uNrOl1aypVdl1R4C2ORq0zdGwJ9vc\nsaIZdeJgsbtPBibv6XrMLLeiS6wzlbY5GrTN0ZCsbU5m19AqoH3M83bhtLjLmFkDYB9gfRJrEhGR\nMpIZBHOBzmbWycz2As4Hyt79fAbwk/Dx2cC/XFeFiYikVNK6hsI+/yuBV4H6wCPuvtjMbgVy3X0G\n8DDwNzNbAXxDEBbJtMfdS3WQtjkatM3RkJRtrnPDUIuISM3KiLGGRESk+hQEIiIRF4kgMLNHzOxr\nM/so3bWkipm1N7M3zWyJmS02s2vSXVOymVmWmb1vZgvDbf5NumtKBTOrb2YfmNlL6a4lFcwsz8w+\nNLMFZpZb9SvqPjNrYWbPmNknZvaxmR1Ro+uPwjECMzsa2Aw87u690l1PKpjZAcAB7j7fzJoB84DT\n3X1JmktLmvCq9CbuvtnMGgKzgWvc/b00l5ZUZvZzIAdo7u6npLueZDOzPCDH3SNzMZmZPQbMcveH\nwrMwG7v7hppafyRaBO7+NsFZSZHh7mvcfX74+DvgY4IruTOWBzaHTxuGPxn9TcfM2gEnAw+luxZJ\nDjPbBzia4CxL3H1HTYYARCQIoi4c1bU/MCe9lSRf2E2yAPga+Ke7Z/o23w38GtiV7kJSyIHXzGxe\nOPxMpusErAUeDbsAHzKzJjX5BgqCDGdmTYFngfHuvind9SSbuxe6ez+CK9kHmVnGdgWa2SnA1+4+\nL921pNhgdx9AMLLxz8Ku30zWABgATHT3/sAWoNyw/ntCQZDBwn7yZ4Ep7v5cuutJpbDp/CYwPN21\nJNGRwIiwz3wqcKyZPZHekpLP3VeFv78GphOMdJzJ8oH8mNbtMwTBUGMUBBkqPHD6MPCxu9+Z7npS\nwcxam1mL8HEj4ATgk/RWlTzu/j/u3s7dswmuyv+Xu1+Q5rKSysyahCc/EHaPDAMy+mxAd/8S+MLM\nuoaTjgNq9KSPOjH66J4ys6eAIUArM8sHbnb3h9NbVdIdCVwIfBj2mQNc7+4z01hTsh0APBbeFKke\nMM3dI3FKZYT8AJge3rakAfCku/8jvSWlxFXAlPCMoZXAxTW58kicPioiIhVT15CISMQpCEREIk5B\nICIScQoCEZGIUxCIiEScgkAkZGaF4YiWRT81dvWmmWVHafRbqVsicR2BSIK2hcNTiESKWgQiVQjH\nv/9DOAb++2Z2SDg928z+ZWaLzOwNM+sQTv+BmU0P74uw0Mx+GK6qvpk9GN4r4bXw6mfM7OrwvhGL\nzGxqmjZTIkxBIFKiUZmuofNi5m10997AXwhG/AS4F3jM3fsAU4B7wun3AP92974EY8IsDqd3Bu5z\n957ABuCscPoEoH+4nrHJ2jiRiujKYpGQmW1296ZxpucBx7r7ynAgvy/dfT8zW0dw85+d4fQ17t7K\nzNYC7dz9+5h1ZBMMi905fH4d0NDdf2tm/yC4cdLzwPMx91QQSQm1CEQS4xU83h3fxzwupOQY3cnA\nfQSth7lmpmN3klIKApHEnBfz+93w8TsEo34CjAJmhY/fAMZB8Y1y9qlopWZWD2jv7m8C1wH7AOVa\nJSLJpG8eIiUaxYzUCvAPdy86hbSlmS0i+FY/Mpx2FcFdo35FcAepohEhrwEmm9mlBN/8xwFrKnjP\n+sATYVgYcE9N34ZQpCo6RiBShSjeLF2iRV1DIiIRpxaBiEjEqUUgIhJxCgIRkYhTEIiIRJyCQEQk\n4hQEIiIR9/8BySQtfkIpF+EAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FImnnPhK0iTg",
        "colab_type": "text"
      },
      "source": [
        "Now normally we would only look at the test set performance once you have your final model, but to show how you do it lets peek now.\n",
        "\n",
        "Remember we saved the model when it improved on the validation set, which is not neccissarily the last epoch because of the chance of overfitting."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WbGF8vra0jyg",
        "colab_type": "code",
        "outputId": "1f90a567-bbf5-421c-a99a-064ca119cf3a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 225
        }
      },
      "source": [
        "clear_session()\n",
        "\n",
        "model = create_model()\n",
        "model.load_weights(best_model_filepath)\n",
        "metrics = model.evaluate(X_test, y_test)\n",
        "\n",
        "pd.DataFrame(metrics, index = model.metrics_names)"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "5050/5050 [==============================] - 1s 221us/sample - loss: 0.0115 - accuracy: 0.9976 - AUC: 0.9981 - Recall: 0.9115 - Precision: 0.9810\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>loss</th>\n",
              "      <td>0.011484</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>accuracy</th>\n",
              "      <td>0.997624</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AUC</th>\n",
              "      <td>0.998093</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Recall</th>\n",
              "      <td>0.911504</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Precision</th>\n",
              "      <td>0.980952</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                  0\n",
              "loss       0.011484\n",
              "accuracy   0.997624\n",
              "AUC        0.998093\n",
              "Recall     0.911504\n",
              "Precision  0.980952"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TY7qQqk1ByQe",
        "colab_type": "text"
      },
      "source": [
        "# Additional Model Parameters\n",
        "\n",
        "Now lets look at some more options to improve our model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oQffQDyx0lfw",
        "colab_type": "text"
      },
      "source": [
        "## Batch Generator\n",
        "\n",
        "Instead of using the fit method we may want to change how batches are made. With this method we do need to specify our own validation set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FrxQhsDj0wRA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "TEST_SIZE = 0.1\n",
        "\n",
        "train_list, test_list = train_test_split(key_list, test_size=TEST_SIZE, \n",
        "                                         random_state=RANDOM_STATE)\n",
        "train_list, val_list = train_test_split(train_list, test_size=TEST_SIZE, \n",
        "                                         random_state=RANDOM_STATE)\n",
        "data_lists = {'train': train_list, \n",
        "              'test': test_list,\n",
        "              'val': val_list,\n",
        "              }\n",
        "\n",
        "data_dict = make_data(data_lists)\n",
        "\n",
        "X_train = data_dict['train'].values\n",
        "X_val = data_dict['val'].values\n",
        "X_test = data_dict['test'].values\n",
        "y_train = data_dict['train'].index.values\n",
        "y_val = data_dict['val'].index.values\n",
        "y_test = data_dict['test'].index.values\n",
        "feat_names = list(data_dict['train'].columns)\n",
        "\n",
        "# lets change integers\n",
        "le = preprocessing.LabelEncoder()\n",
        "y_train = le.fit_transform(y_train)\n",
        "y_val = le.transform(y_val)\n",
        "y_test = le.transform(y_test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PiPG_vD0z0I_",
        "colab_type": "text"
      },
      "source": [
        "There are some useful batch generators that are easy to impliment. As we are using an imballanced dataset, we can use an already made one to ballance it up."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fBPwB6Ts0yWu",
        "colab_type": "code",
        "outputId": "0c4dac4b-2b92-4f49-97bf-0986f7628fd7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 189
        }
      },
      "source": [
        "from imblearn.tensorflow import balanced_batch_generator\n",
        "from imblearn.under_sampling import NearMiss\n",
        "from imblearn.combine import SMOTEENN\n",
        "from imblearn.over_sampling import RandomOverSampler\n",
        "training_generator, steps_per_epoch = balanced_batch_generator(X_train, \n",
        "                                                               y_train, \n",
        "                                                               sampler=RandomOverSampler(), \n",
        "                                                               batch_size=BATCH_SIZE, \n",
        "                                                               random_state=RANDOM_STATE)"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/externals/six.py:31: FutureWarning: The module is deprecated in version 0.21 and will be removed in version 0.23 since we've dropped support for Python 2.7. Please rely on the official version of six (https://pypi.org/project/six/).\n",
            "  \"(https://pypi.org/project/six/).\", FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.neighbors.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.neighbors. Anything that cannot be imported from sklearn.neighbors is now part of the private API.\n",
            "  warnings.warn(message, FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.utils.testing module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.utils. Anything that cannot be imported from sklearn.utils is now part of the private API.\n",
            "  warnings.warn(message, FutureWarning)\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a3_NgJQT0vag",
        "colab_type": "text"
      },
      "source": [
        "Now we can create the model as before except we use the fit_generator instead of just fit\n",
        "\n",
        "**TODO**\n",
        "- update to just fit as this has changed"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YTKD5yRo1Qxn",
        "colab_type": "code",
        "outputId": "6922be4c-401e-4340-b783-503b9a5194d5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 551
        }
      },
      "source": [
        "from math import ceil\n",
        "\n",
        "# clear the previous keras model\n",
        "clear_session()\n",
        "\n",
        "model = create_model()\n",
        "\n",
        "callbacks = create_callbacks('Best_bal_MLP_Model.ckpt',\n",
        "                            './bal_MLP_logs/')\n",
        "\n",
        "# This is the different bit\n",
        "# =======================================================\n",
        "history = model.fit_generator(generator=training_generator,\n",
        "                              epochs=EPOCHS,\n",
        "                              steps_per_epoch=steps_per_epoch,\n",
        "                              validation_data = (X_val, y_val),\n",
        "                              callbacks=callbacks,\n",
        "                              verbose=1)\n",
        "# ======================================================="
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From <ipython-input-37-3b2725489fa6>:18: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use Model.fit, which supports generators.\n",
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n",
            "Train for 439 steps, validate on 2411 samples\n",
            "Epoch 1/20\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "439/439 [==============================] - 5s 10ms/step - loss: 0.0282 - accuracy: 0.9907 - AUC: 0.9988 - Recall: 0.9935 - Precision: 0.9879 - val_loss: 0.0902 - val_accuracy: 0.9805 - val_AUC: 0.9814 - val_Recall: 0.8408 - val_Precision: 0.9185\n",
            "Epoch 2/20\n",
            "433/439 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9990 - AUC: 0.9998 - Recall: 0.9994 - Precision: 0.9985\n",
            "Epoch 00002: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.\n",
            "439/439 [==============================] - 3s 7ms/step - loss: 0.0044 - accuracy: 0.9990 - AUC: 0.9998 - Recall: 0.9994 - Precision: 0.9985 - val_loss: 0.1110 - val_accuracy: 0.9826 - val_AUC: 0.9715 - val_Recall: 0.8706 - val_Precision: 0.9162\n",
            "Epoch 3/20\n",
            "433/439 [============================>.] - ETA: 0s - loss: 9.1985e-04 - accuracy: 0.9998 - AUC: 1.0000 - Recall: 1.0000 - Precision: 0.9996\n",
            "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0001.\n",
            "439/439 [==============================] - 3s 7ms/step - loss: 9.0863e-04 - accuracy: 0.9998 - AUC: 1.0000 - Recall: 1.0000 - Precision: 0.9996 - val_loss: 0.1386 - val_accuracy: 0.9772 - val_AUC: 0.9497 - val_Recall: 0.7910 - val_Precision: 0.9244\n",
            "Epoch 4/20\n",
            "439/439 [==============================] - 3s 8ms/step - loss: 1.5714e-04 - accuracy: 1.0000 - AUC: 1.0000 - Recall: 1.0000 - Precision: 0.9999 - val_loss: 0.1417 - val_accuracy: 0.9768 - val_AUC: 0.9522 - val_Recall: 0.7861 - val_Precision: 0.9240\n",
            "Epoch 5/20\n",
            "439/439 [==============================] - 3s 7ms/step - loss: 8.1661e-05 - accuracy: 1.0000 - AUC: 1.0000 - Recall: 1.0000 - Precision: 1.0000 - val_loss: 0.1464 - val_accuracy: 0.9768 - val_AUC: 0.9448 - val_Recall: 0.7861 - val_Precision: 0.9240\n",
            "Epoch 6/20\n",
            "439/439 [==============================] - 3s 7ms/step - loss: 5.6630e-05 - accuracy: 1.0000 - AUC: 1.0000 - Recall: 1.0000 - Precision: 1.0000 - val_loss: 0.1506 - val_accuracy: 0.9772 - val_AUC: 0.9448 - val_Recall: 0.7910 - val_Precision: 0.9244\n",
            "Epoch 00006: early stopping\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xtkXALVRJxaM",
        "colab_type": "text"
      },
      "source": [
        "You can see there is still poor overfitting."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rckqvh1P1Tl-",
        "colab_type": "code",
        "outputId": "d2a30f3a-661e-4767-eb87-79e9cd530327",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "plot_progress(history.history)"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5xVdb3/8debAUQuAgKWchFKU0BR\ncFITFVFDTIUsUhC8pZGmUZklZVnasbRjSirHwruCogd/Jl6QOmmR5xgykGKIKCrqiBdAQRG8DHx+\nf6w1sBnWDAPMnj0z+/18PPaDvb7rsj9rz4P12d/vd63vVxGBmZlZVc0KHYCZmTVMThBmZpbJCcLM\nzDI5QZiZWSYnCDMzy+QEYWZmmZwgrF5IKpG0WlKPuty2kCTtISkv94lXPbakP0sanY84JP1c0h+2\ndf8ajnu2pL/V9XGt/jhBWKb0Al35Wi9pbc5y5oWqJhGxLiLaRsRrdbltQyXpfyRdklH+dUlvSCrZ\nmuNFxJCImFIHcR0taUmVY/8qIs7Z3mNb0+MEYZnSC3TbiGgLvAackFO22YVKUvP6j7JBux04NaP8\nVGByRKyr53jMtpoThG0TSf8h6R5Jd0v6ABgj6UuS/ilppaQ3JV0rqUW6fXNJIalnujw5XT9D0geS\nnpTUa2u3TdcfK+kFSaskXSfpfyWdUU3ctYnx25IWS3pP0rU5+5ZIukbSCkkvA0Nr+Ir+H/BZSYfk\n7N8J+ApwR7o8TNLTkt6X9Jqkn9fwfT9ReU5biiNt2lmYflcvSTo7LW8PPAj0yKkN7pL+LW/L2f9E\nSQvS7+gxSXvlrCuXdIGkZ9Pv+25JO9TwPeTGdaiksnS/pyQdlLPuLElL0phfljQyLf+CpFnpPssl\n3VWbz7I6EhF++VXjC1gCHF2l7D+AT4ATSH5o7Ah8ETgIaA58DngBOD/dvjkQQM90eTKwHCgFWgD3\nkPyy3tptdwE+AIan6y4APgXOqOZcahPjA0B7oCfwbuW5A+cDC4BuQCdgVvJfqNrv7VbgDznL5wFl\nOctHAn3T72+/9ByPT9ftkXts4InKc9pSHOnf5HOA0s9YC/RL1x0NLMn4W96Wvu8NrE73awH8FFgE\ntEjXlwP/BD6bfvYLwNnVnP/ZwN/S952BVcCo9Hs+FVgBdAR2StftmW67K9Anff/fwEXpd9QKGFjo\n/w/F9HINwrbHExHxYESsj4i1ETEnImZHREVEvAxMAgbVsP+0iCiLiE+BKcD+27Dt8cDTEfFAuu4a\nkgttplrG+JuIWBURS4C/5XzWScA1EVEeESuAK2qIF5JmppNyfmGflpZVxvJYRCxIv79ngKkZsWSp\nMY70b/JyJB4D/gocVovjAowEpqexfZoeuz1JUq00ISLeSj/7IWr+u1U6AVgQEXen3/2dwMvAcZVh\nA/tIahURb0bEc2n5pySJeteI+Cgi/reW52F1wAnCtsfruQuS9pb0sKS3JL0PXEbyy7E6b+W8XwO0\n3YZtd8uNIyKC5FduplrGWKvPAl6tIV6AvwPvAydI+gLQH7g7J5YvSfqbpGWSVpH84q7p+6pUYxyS\njpc0W9K7klYCQ2p53MpjbzheRKwn+T675myzNX+3zOPmxN01It4nqVmcB7wl6aH0+wL4IUlNpixt\n1jq9ludhdcAJwrZH1Vsr/wj8G9gjInYCLiFp5sinN0maWgCQJDa9mFW1PTG+CXTPWa7xNtw0Wd1B\nUnM4FXgkInJrN1OB+4DuEdEeuKmWsVQbh6QdgWnAb4DPREQH4M85x93S7bBLgd1zjteM5Pt9oxZx\n1fq4qR6Vx42IGRFxNEnz0mKSvxNpbeLsiNiVJIFMyu1/svxygrC61I6kLflDSb2Bb9fDZz4EDJB0\ngpI7qb4HdMlTjPcC35fUNe1wvqgW+9xB0on8TXKal3JieTciPpJ0MEnzzvbGsQPQElgGrJN0PHBU\nzvq3gc6S2tVw7GGSjkg7739E0sczu5axVechoK+kk9ObAU4h6Wd5WNKu6d+vNUm/1ofAegBJJ0mq\nTPgrSRKc7wCrJ04QVpd+CJxOckH5I0lncl5FxNvAycDVJJ2enwf+BXychxhvIGnPfxaYQ/JLfUvx\nLQaeIrlwP1xl9bnAb5TcBfZTkovzdsURESuBHwD3k3SwjyC5OFeu/zdJrWVJepfSLlXiXUDy/dxA\nkmSGAsPS/ohtFhHLgGEkyWxFGuPxEfEeUEKSiN5M1x1CUluApO9jjqQPSe4MOy8a8fMxjY2SWrBZ\n06DkAbSlwIiI+Eeh4zFrzFyDsEZP0lBJHdK7hX5OcufLUwUOy6zRc4KwpuBQklsmlwHHACdGRHVN\nTGZWS25iMjOzTK5BmJlZpiYzwFrnzp2jZ8+ehQ7DzKxRmTt37vKIyLw1vMkkiJ49e1JWVlboMMzM\nGhVJ1Y4I4CYmMzPL5ARhZmaZnCDMzCxTk+mDyPLpp59SXl7ORx99VOhQrBZatWpFt27daNGiRaFD\nMTOaeIIoLy+nXbt29OzZk2SQT2uoIoIVK1ZQXl5Or14erNOsIWjSTUwfffQRnTp1cnJoBCTRqVMn\n1/bMGpAmnSAAJ4dGxH8rs4alSTcxmZk1JZ9+Cu+8A2+/vemrY0cYO7buP88JIo9WrFjBUUclc7W8\n9dZblJSU0KVL8sDiU089RcuWLbd4jDPPPJPx48ez1157VbvNxIkT6dChA6NHj97umA899FCuv/56\n9t+/NtMMm9n2+vjjzS/41b3efTf7GF/6khNE3k2ZAhdfDK+9Bj16wOWXw/Zcczt16sTTTz8NwC9/\n+Uvatm3LhRdeuMk2EUFE0KxZdmvfrbfeusXPOe+887a4jZnVn7VrN17U33qr5ov+qlXZx9hpJ/jM\nZ5JXnz4wePDG5aqvNm3ycx5OEKkpU5IMvGZNsvzqqxszch38MN/E4sWLGTZsGP379+df//oXf/nL\nX7j00kuZN28ea9eu5eSTT+aSSy4BNv6i32effejcuTPnnHMOM2bMoHXr1jzwwAPssssu/OxnP6Nz\n5858//vf59BDD+XQQw/lscceY9WqVdx6660ccsghfPjhh5x22mksXLiQPn36sGTJEm666aYaawqT\nJ0/myiuvJCIYNmwYv/71r6moqODMM8/k6aefJiIYO3Ys48aN45prruHGG2+kefPm9OvXj8mTJ9ft\nl2ZWYKtX1/6X/gcfZB+jQ4eNF/X99oPPfjb7gr/LLrDjjvV7flmcIFIXX7wxOVRasyYpr+sEAfD8\n889zxx13UFpaCsAVV1zBzjvvTEVFBYMHD2bEiBH06dNnk31WrVrFoEGDuOKKK7jgggu45ZZbGD9+\n/GbHjgieeuoppk+fzmWXXcajjz7Kddddx2c/+1nuu+8+nnnmGQYMGFBjfOXl5fzsZz+jrKyM9u3b\nc/TRR/PQQw/RpUsXli9fzrPPPgvAypUrAfjtb3/Lq6++SsuWLTeUmTVkEfD++7W/6Fe9PlTq1Gnj\nhb20tPpf+bvsAjvsUL/nuL2cIFKvVTPLbXXl2+vzn//8huQAcPfdd3PzzTdTUVHB0qVLee655zZL\nEDvuuCPHHnssAAcccAD/+Ef2jJpf+9rXNmyzZMkSAJ544gkuuiiZ236//fajb9++NcY3e/Zsjjzy\nSDp37gzAKaecwqxZs7joootYtGgR48aN47jjjmPIkCEA9O3blzFjxjB8+HC++tWvbuW3YVY3ImDl\nyi0361S+Ps6YVkqCLl02Xtg///nqL/pdukBTfq7TCSLVo0fSrJRVng9tchoNX3zxRX7/+9/z1FNP\n0aFDB8aMGZP5PEBup3ZJSQkVFRWZx94h/ZlS0zbbqlOnTsyfP58ZM2YwceJE7rvvPiZNmsTMmTP5\n+9//zvTp0/n1r3/N/PnzKSkpqdPPtuIVkTTbLF265VfWRb+kZONF/7Ofhd69q7/od+6cbG9OEBtc\nfvmmfRAArVsn5fn2/vvv065dO3baaSfefPNNZs6cydChQ+v0MwYOHMi9997LYYcdxrPPPstzzz1X\n4/YHHXQQF154IStWrKB9+/ZMnTqVCy+8kGXLltGqVSu+8Y1vsOeee3L22Wezbt06ysvLOfLIIzn0\n0EPp3r07a9asoV27dnV6DtY0rVkDb75Z/QX/jTeSfz/8cPN927WD3XZLXocckvy7666bX/Q7dYJq\n7gOxGjhBpCr7GeryLqbaGjBgAH369GHvvfdm9913Z+DAgXX+Gd/97nc57bTT6NOnz4ZX+/btq92+\nW7du/OpXv+KII44gIjjhhBM47rjjmDdvHmeddRYRgSSuvPJKKioqOOWUU/jggw9Yv349F154oZOD\n8cknSVNPdRf8yldWl1WrVtC1a3LBHzAAjj9+YyKofO26a5IgLH+azJzUpaWlUXXCoIULF9K7d+8C\nRdSwVFRUUFFRQatWrXjxxRcZMmQIL774Is2bN6zfCP6bNXzr1iUPa1W90Fd9LVu2+b4tWiQX9qoX\n+6qvDh2SvgDLP0lzI6I0a13DujpY3qxevZqjjjqKiooKIoI//vGPDS45WGGtXw8rVmy5jf+tt5Jt\nczVrljTl7LZbUvs++OCNNYDcl5t6GhdfIYpEhw4dmDt3bqHDsAKISB7Gqumi/8YbST/Ap59uvn+X\nLhsv8Pvtl/2Lf5ddwL83mp68/kklDQV+D5QAN0XEFVXWHw5MAPoBIyNiWpX1OwHPAX+KiPPzGatZ\nQ1d5J89779X+VVkjWLt28+N16LDxAn/EEZtf9Lt2Te74qcWIMNZE5S1BSCoBJgJfBsqBOZKmR0Tu\n7TOvAWcAF25+BAB+BczKV4xm9a3y4aytuchXvlauTNr/q1NSkgzalvvaY4/sX/y77prcpWdWk3zW\nIA4EFkfEywCSpgLDSWoEAETEknTd+qo7SzoA+AzwKJDZgWJWCOvXZ/+Sf/fd2l3kq7bf56p6kd95\n5+RBraoX/qqvnXeGtm3dsWt1K58Joivwes5yOXBQbXaU1Az4HTAGOLqG7cYCYwF65OuJNmuS1q/f\nvl/yNV3kmzff9OLduTPsueeWL/IdO/oibw1LQ+1W+g7wSESU1zSJTERMAiZBcptrPcVWa4MHD2b8\n+PEcc8wxG8omTJjAokWLuOGGG6rdr23btqxevZqlS5cybtw4pk2bttk2RxxxBFddddUmw3VUNWHC\nBMaOHUvrtC3hK1/5CnfddRcdOnTYjrOqfmTahmr9erjzTrjhhuTWy8qLfE13eLdosemFu0sX+MIX\naneRb9PGF3lrGvKZIN4Auucsd0vLauNLwGGSvgO0BVpKWh0Rm49M14CNGjWKqVOnbpIgpk6dym9/\n+9ta7b/bbrtlJofamjBhAmPGjNmQIB555JFtPlZjNW8enH8+PPlkcgfOl75Uu4t869a+yJvl847k\nOcCeknpJagmMBKbXZseIGB0RPSKiJ0kH9h2NLTkAjBgxgocffphPPvkEgCVLlrB06VIOO+ywDc8l\nDBgwgH333ZcHHnhgs/2XLFnCPvvsA8DatWsZOXIkvXv35sQTT2Rtzm0p5557LqWlpfTt25df/OIX\nAFx77bUsXbqUwYMHM3jwYAB69uzJ8uXLAbj66qvZZ5992GeffZgwYcKGz+vduzff+ta36Nu3L0OG\nDNnkc7I8/fTTHHzwwfTr148TTzyR9957b8Pn9+nTh379+jFy5EgA/v73v7P//vuz//77079/fz6o\nbkzkOvDuu/Cd7ySjay5eDLfckiSLyZPhuuvgssvgBz+AM86A4cPh8MNh332hWzfXAMwq5a0GEREV\nks4HZpLc5npLRCyQdBlQFhHTJX0RuB/oCJwg6dKIqHmY0W30/e9DOndPndl/f0ivrZl23nlnDjzw\nQGbMmMHw4cOZOnUqJ510EpJo1aoV999/PzvttBPLly/n4IMPZtiwYdXOy3zDDTfQunVrFi5cyPz5\n8zcZrvvyyy9n5513Zt26dRx11FHMnz+fcePGcfXVV/P4449vGJG10ty5c7n11luZPXs2EcFBBx3E\noEGD6NixIy+++CJ33303N954IyeddBL33XcfY8aMqfYcTzvtNK677joGDRrEJZdcwqWXXsqECRO4\n4ooreOWVV9hhhx02DP991VVXMXHiRAYOHMjq1atp1arVVnzbtbNuXZIMfvKTpCnp/POTZLCdrWpm\nRSmvzzRGxCMR8YWI+HxEXJ6WXRIR09P3cyKiW0S0iYhOWckhIm5rzM9AVDYzQdK8NGrUKCCZs+Gn\nP/0p/fr14+ijj+aNN97g7bffrvY4s2bN2nCh7tevH/369duw7t5772XAgAH079+fBQsWbHEgviee\neIITTzyRNm3a0LZtW772ta9tGDq8V69eGyYRyh0uPMuqVatYuXIlgwYNAuD0009n1qxZG2IcPXo0\nkydP3vDE9sCBA7ngggu49tprWblyZZ0/yT17dvIE79ixyQxc//oXXHutk4PZtmqondR1rqZf+vk0\nfPhwfvCDHzBv3jzWrFnDAQccAMCUKVNYtmwZc+fOpUWLFvTs2TNziO8teeWVV7jqqquYM2cOHTt2\n5Iwzztim41TaIWdGk5KSki02MVXn4YcfZtasWTz44INcfvnlPPvss4wfP57jjjuORx55hIEDBzJz\n5kz23nvvbY610rJlMH58UnPYdddkdsBRo9xMZLa9PCpKnrVt25bBgwfzzW9+c0PtAZJf37vssgst\nWrTg8ccf59WsyShyHH744dx1110A/Pvf/2b+/PlAMlR4mzZtaN++PW+//TYzZszYsE+7du0y2/kP\nO+ww/vSnP7FmzRo+/PBD7r//fg477LCtPrf27dvTsWPHDbWPO++8k0GDBrF+/Xpef/11Bg8ezJVX\nXsmqVatYvXo1L730Evvuuy8XXXQRX/ziF3n++ee3+jNzVVTA9dcndxfdcQdceCE8/zyccoqTg1ld\nKJoaRCGNGjWKE088cUNTE8Do0aM54YQT2HfffSktLd3iL+lzzz2XM888k969e9O7d+8NNZH99tuP\n/v37s/fee9O9e/dNhgofO3YsQ4cOZbfdduPxxx/fUD5gwADOOOMMDjzwQADOPvts+vfvX2NzUnVu\nv/12zjnnHNasWcPnPvc5br31VtatW8eYMWNYtWoVEcG4cePo0KEDP//5z3n88cdp1qwZffv23TA7\n3rZ44omkf+GZZ+Coo5KOZw8Ca1a3PNy3NShb+pu9+Sb8+MfJ3Ujdu8PVV8PXv+4ag9m2qmm4bzcx\nWaPw6adJMthrL7j33mRip4ULYcQIJwezfHETkzV4jz2WNCctXAjHHgu//30ydIWZ5VeTr0E0lSa0\nYlD1b/X663DyyUkfw0cfwQMPwMMPOzmY1ZcmnSBatWrFihUrnCQagYhgxYoVtGrVio8/hiuugL33\nhunT4dJLYcECGDbMzUlm9alJNzF169aN8vJylmVNjmsNTqtWrXjhhW4ccwy8+CJ89atwzTXQs2eh\nIzMrTk06QbRo0YJevXoVOgyrhVdeScZGeuCBpAlpxgwYOrTQUZkVtybdxGQN39q1SRNSnz7wl7/A\nb34Dzz7r5GDWEDTpGoQ1XBHw4IPJIIqvvAInnQRXXZU822BmDYNrEFbvXnwRjjsuGWZ7xx3hr3+F\ne+5xcjBraJwgrN58+GHygNs++yRDZVx9dTIE+5FHFjoyM8viJibLuwiYNg0uuADKy+HUU+HKK5OR\nV82s4XINwvJq4UL48peTPoadd4Z//CMZedXJwazhc4KwvPjgA/jRj6BfP5g7NxmWe+5cOPTQQkdm\nZrWV1wQhaaikRZIWS9psTmlJh0uaJ6lC0oic8v0lPSlpgaT5kk7OZ5xWdyKSCXv22iu5K+n00+GF\nF+C886COJ5AzszzLW4KQVAJMBI4F+gCjJPWpstlrwBnAXVXK1wCnpVOQDgUmSPLEkQ3c/PkwaBCM\nGQNduyZTgN50E3TpUujIzGxb5LMGcSCwOCJejohPgKnA8NwNImJJRMwH1lcpfyEiXkzfLwXeAXyZ\naaBWroRx46B/f3juOZg0Cf75T0jnIzKzRiqfCaIr8HrOcnlatlUkHQi0BF7KWDdWUpmkMo+3VP/W\nr4dbb02m/Jw4Ec45J2lO+ta3oKSk0NGZ2fZq0J3UknYF7gTOjIj1VddHxKSIKI2I0i5ux6hXZWVw\nyCHwzW/CHnskyxMnJncqmVnTkM8E8QaQ+2xst7SsViTtBDwMXBwR/6zj2GwbrVgB3/520ny0ZAnc\nfnvy0Fv//oWOzMzqWj4TxBxgT0m9JLUERgLTa7Njuv39wB0RMS2PMVotrVsHf/hD0px0883wve/B\nokVw2mnQrEHXQ81sW+Xtv3ZEVADnAzOBhcC9EbFA0mWShgFI+qKkcuAbwB8lLUh3Pwk4HDhD0tPp\na/98xWo1e/LJpMZw7rmw777J8BjXXAPt2xc6MjPLJzWV2dZKS0ujrKys0GE0KW+/DePHw223wW67\nwe9+l0wB6lndzJoOSXMjojRrnRsHbDMVFXDttcnDblOmwI9/nDQnjRzp5GBWTPxsq21i1iw4//xk\n0p4vfzlJFHvvXeiozKwQXIMwAJYuhdGjkyehV62C++6DmTOdHMyKmRNEkfvkk2TMpL32SpLCz3+e\njMD6ta+5Ocms2LmJqYj9z//Ad78Lzz8Pxx8PEybA5z9f6KjMrKFwDaIIzZ8PI0YkfQyffJLMDf3g\ng04OZrYpJ4giEZH0KQwZAvvtBzNmwK9+BQsWJLUHM7Oq3MTUxH38Mdx1VzL/87//nczk9utfJ8Nl\neNwkM6uJE0QTtWJFMjTG9dfDW28lT0DfdhuMGgUtWxY6OjNrDJwgmpjFi5NhMG67DdasgWOOSeaA\nPvpo35VkZlvHCaIJiID/+7/kdtUHHoAWLZJnGi64APbZp9DRmVlj5QTRiFVUwP33J2MkzZ6d9Cn8\n9KfJ/M+77lro6MyssXOCaIQ++ABuuSV5bmHJkuT21OuvhzPOgDZtCh2dmTUVThCNSHk5XHcd/PGP\nyXAYAwcmdycNG+YpPs2s7jlBNAJPP500I02dmswD/fWvww9/CAcdVOjIzKwpc4JooNavh0cfTRLD\nY48lTUfnnZfM5NarV6GjM7Ni4ATRwHz0UTIHw+9+lwya17UrXHkljB0LHToUOjozKyZOEA3E8uVw\nww1JZ/M778D++8Odd8JJJ/nBNjMrjLyOxSRpqKRFkhZLGp+x/nBJ8yRVSBpRZd3pkl5MX6fnM85C\neuGFZK7nHj3gkkugtBT++leYNw/GjHFyMLPCyVsNQlIJMBH4MlAOzJE0PSKey9nsNeAM4MIq++4M\n/AIoBQKYm+77Xr7irU8R8I9/JM1IDz6YPNh26qnJg219+hQ6OjOzRD6bmA4EFkfEywCSpgLDgQ0J\nIiKWpOvWV9n3GOAvEfFuuv4vwFDg7jzGm3cVFTBtWpIYysqgUyf42c+SzufPfKbQ0ZmZbSqfCaIr\n8HrOcjlQ2xszs/btWnUjSWOBsQA9evTYtijrwfvvw003we9/D6+9BnvumfQ3nHYatG5d6OjMzLI1\n6k7qiJgETAIoLS2NAoezmddfh2uvhUmTkiRx+OHJg27HHw/NPBOHmTVw+UwQbwDdc5a7pWW13feI\nKvv+rU6iqgfz5iXNSPfem/Q3fOMbyYNtpaWFjszMrPby+Tt2DrCnpF6SWgIjgem13HcmMERSR0kd\ngSFpWYO1fj089BAMHgwHHJB0Po8bBy+9BHff7eRgZo1P3moQEVEh6XySC3sJcEtELJB0GVAWEdMl\nfRG4H+gInCDp0ojoGxHvSvoVSZIBuKyyw7qhWbs2eV7h6qth0SLo1g3+8z/hW9+C9u0LHZ2Z2bZT\nRINrut8mpaWlUVZWVm+f98478F//lbyWLYMBA5JmpG98I7lt1cysMZA0NyIy2zgadSd1ITz/fFJb\nuOOOZL7n449PEsOgQZ6xzcyaFieIWoiAv/0t6Xh++GFo1QpOPx1+8APYe+9CR2dmlh9OEDX49FP4\n7/9OEsO8edClC/zyl/Cd7yTvzcyaMieIDKtWwY03Jg+2lZcntYRJk5KxkXbcsdDRmZnVDyeIHK++\nmiSFm25KpvUcPBj+8Ac49lg/2GZmxccJApgzJ2lGmjYtWT755KTjecCAwsZlZlZIRZ8gFi+GAw+E\nnXZKOp3HjYPu3be8n5lZU1f0CWKPPZIhMY45JkkSZmaWKPoEAcnDbWZmtil3vZqZWSYnCDMzy+QE\nYWZmmZwgzMwskxOEmZllcoIwM7NMThBmZpbJCcLMzDLlNUFIGippkaTFksZnrN9B0j3p+tmSeqbl\nLSTdLulZSQsl/SSfcZqZ2ebyliAklQATgWOBPsAoSX2qbHYW8F5E7AFcA1yZln8D2CEi9gUOAL5d\nmTzMzKx+5LMGcSCwOCJejohPgKnA8CrbDAduT99PA46SJCCANpKaAzsCnwDv5zFWMzOrolYJQtL3\nJO2kxM2S5kkasoXdugKv5yyXp2WZ20REBbAK6ESSLD4E3gReA66KiHcz4horqUxS2bJly2pzKmZm\nVku1rUF8MyLeB4YAHYFTgSvyFlVS+1gH7Ab0An4o6XNVN4qISRFRGhGlXTwHqJlZnaptglD671eA\nOyNiQU5Zdd4AcmdW6JaWZW6TNie1B1YApwCPRsSnEfEO8L9AaS1jNTOzOlDbBDFX0p9JEsRMSe2A\n9VvYZw6wp6RekloCI4HpVbaZDpyevh8BPBYRQdKsdCSApDbAwcDztYzVzMzqQG3ngzgL2B94OSLW\nSNoZOLOmHSKiQtL5wEygBLglIhZIugwoi4jpwM3AnZIWA++SJBFI7n66VVJlTeXWiJi/tSdnZmbb\nTskP9i1sJA0Eno6IDyWNAQYAv4+IV/MdYG2VlpZGWVlZocMwM2tUJM2NiMwm/No2Md0ArJG0H/BD\n4CXgjjqKz8zMGqDaJoiKtG9gOHB9REwE2uUvLDMzK7Ta9kF8kA53cSpwmKRmQIv8hWVmZoVW2xrE\nycDHJM9DvEVyy+p/5i0qMzMruFoliDQpTAHaSzoe+Cgi3AdhZtaE1XaojZOAp0gG0TsJmC1pRD4D\nMzOzwqptH8TFwBfTp5qR1AX4H5Ixk8zMrAmqbR9Es8rkkFqxFfuamVkjVNsaxKOSZgJ3p8snA4/k\nJyQzM2sIapUgIuJHkr4ODOue+ScAAA0oSURBVEyLJkXE/fkLy8zMCq22NQgi4j7gvjzGYmZmDUiN\nCULSBySzu222CoiI2CkvUZmZWcHVmCAiwsNpmJkVKd+JZGZmmZwgzMwskxOEmZllcoIwM7NMeU0Q\nkoZKWiRpsaTxGet3kHRPun62pJ456/pJelLSAknPSmqVz1jNzGxTeUsQkkpI5pY+FugDjJLUp8pm\nZwHvRcQewDXAlem+zYHJwDkR0Rc4Avg0X7Gamdnm8lmDOBBYHBEvR8QnwFSSGelyDQduT99PA46S\nJGAIMD8ingGIiBURsS6PsZqZWRX5TBBdgddzlsvTssxtIqICWAV0Ar4AhKSZkuZJ+nHWB0gaK6lM\nUtmyZcvq/ATMzIpZQ+2kbg4cCoxO/z1R0lFVN4qISRFRGhGlXbp0qe8YzcyatHwmiDeA7jnL3dKy\nzG3Sfof2JEOJlwOzImJ5RKwhGTl2QB5jNTOzKvKZIOYAe0rqJaklMBKYXmWb6cDp6fsRwGMREcBM\nYF9JrdPEMQh4Lo+xmplZFbUezXVrRUSFpPNJLvYlwC0RsUDSZUBZREwHbgbulLQYeJckiRAR70m6\nmiTJBPBIRDycr1jNzGxzSn6wN36lpaVRVlZW6DDMzBoVSXMjojRrXUPtpDYzswJzgjAzs0xOEGZm\nlskJwszMMjlBmJlZJicIMzPL5ARhZmaZnCDMzCyTE4SZmWVygjAzs0xOEGZmlskJwszMMjlBmJlZ\nJicIMzPL5ARhZmaZnCDMzCyTE4SZmWXKa4KQNFTSIkmLJY3PWL+DpHvS9bMl9ayyvoek1ZIuzGec\nZma2ubwlCEklwETgWKAPMEpSnyqbnQW8FxF7ANcAV1ZZfzUwI18xmplZ9fJZgzgQWBwRL0fEJ8BU\nYHiVbYYDt6fvpwFHSRKApK8CrwAL8hijmZlVI58Joivwes5yeVqWuU1EVACrgE6S2gIXAZfW9AGS\nxkoqk1S2bNmyOgvczMwabif1L4FrImJ1TRtFxKSIKI2I0i5dutRPZGZmRaJ5Ho/9BtA9Z7lbWpa1\nTbmk5kB7YAVwEDBC0m+BDsB6SR9FxPV5jNfMzHLkM0HMAfaU1IskEYwETqmyzXTgdOBJYATwWEQE\ncFjlBpJ+Cax2cjAzq195SxARUSHpfGAmUALcEhELJF0GlEXEdOBm4E5Ji4F3SZKImZk1AEp+sDd+\npaWlUVZWVugwzMwaFUlzI6I0a11D7aQ2M7MCc4IwM7NMThBmZpbJCcLMzDI5QZiZWSYnCDMzy+QE\nYWZmmYo+QUyZAj17QrNmyb9TphQ6IjOzhiGfQ200eFOmwNixsGZNsvzqq8kywOjRhYvLzKwhKOoa\nxMUXb0wOldasScrNzIpdUSeI117bunIzs2JS1AmiR4+tKzczKyZFnSAuvxxat960rHXrpNzMrNgV\ndYIYPRomTYLddwcp+XfSJHdQm5lBkd/FBEkycEIwM9tcUdcgzMysek4QZmaWyQnCzMwy5TVBSBoq\naZGkxZLGZ6zfQdI96frZknqm5V+WNFfSs+m/R+YzTjMz21zeEoSkEmAicCzQBxglqU+Vzc4C3ouI\nPYBrgCvT8uXACRGxL3A6cGe+4jQzs2z5rEEcCCyOiJcj4hNgKjC8yjbDgdvT99OAoyQpIv4VEUvT\n8gXAjpJ2yGOsZmZWRT4TRFfg9Zzl8rQsc5uIqABWAZ2qbPN1YF5EfFz1AySNlVQmqWzZsmV1FriZ\nmTXwTmpJfUmanb6dtT4iJkVEaUSUdunSpX6DMzNr4vKZIN4Auucsd0vLMreR1BxoD6xIl7sB9wOn\nRcRLeYzTzMwy5DNBzAH2lNRLUktgJDC9yjbTSTqhAUYAj0VESOoAPAyMj4j/zWOMZmZWjbwliLRP\n4XxgJrAQuDciFki6TNKwdLObgU6SFgMXAJW3wp4P7AFcIunp9LVLvmI1M7PNKSIKHUOdKC0tjbKy\nskKHYWbWqEiaGxGlWesadCe1mZkVjhOEmZllcoIwM7NMThBmZpbJCcLMzDI5QZiZWSYnCDMzy+QE\nYWZmmZwgzMwskxOEmZllcoIwM7NMThBFaMoU6NkTmjVL/p0ypdARmVlD1LzQAVj9mjIFxo6FNWuS\n5VdfTZYBRo8uXFxm1vC4BlFkLr54Y3KotGZNUm5mlssJosi89trWlZtZ8XKCKDI9emxduZkVLyeI\nInP55dC69aZlrVsn5U2VO+XNtk1eE4SkoZIWSVosaXzG+h0k3ZOuny2pZ866n6TliyQdk884i8no\n0TBpEuy+O0jJv5MmNd0O6spO+VdfhYiNnfJNPUkUY1L0OefhnCMiLy+gBHgJ+BzQEngG6FNlm+8A\nf0jfjwTuSd/3SbffAeiVHqekps874IADwqyq3XePSFLDpq/ddy90ZPkzeXJE69abnm/r1kl5U+Vz\n3vZzBsqimutq3uaklvQl4JcRcUy6/JM0If0mZ5uZ6TZPSmoOvAV0Acbnbpu7XXWf5zmpLUuzZsl/\nnaokWL++/uOpDz17JjWlqnbfHZYsqe9o6ofPeaOtPedCzUndFXg9Z7k8LcvcJiIqgFVAp1rui6Sx\nksoklS1btqwOQ7emohg75YvxTjWf85bLt0Wj7qSOiEkRURoRpV26dCl0ONYAFWOnfDEmRZ/zlsu3\nRT4TxBtA95zlbmlZ5jZpE1N7YEUt9zXbomLrlIfiTIo+50Sdn3N1nRPb+yIZxuNlkk7myk7qvlW2\nOY9NO6nvTd/3ZdNO6pdxJ7VZrU2enHTES8m/TbmztpLPedvOmUJ0UgNI+gowgeSOplsi4nJJl6UB\nTZfUCrgT6A+8C4yMiJfTfS8GvglUAN+PiBk1fZY7qc3Mtl5NndR5TRD1yQnCzGzrFeouJjMza8Sc\nIMzMLJMThJmZZXKCMDOzTE2mk1rSMiDjwfNa6wwsr6NwGotiO+diO1/wOReL7Tnn3SMi80njJpMg\ntpeksup68puqYjvnYjtf8DkXi3yds5uYzMwskxOEmZllcoLYaFKhAyiAYjvnYjtf8DkXi7ycs/sg\nzMwsk2sQZmaWyQnCzMwyFX2CkHSLpHck/bvQsdQHSd0lPS7pOUkLJH2v0DHlm6RWkp6S9Ex6zpcW\nOqb6IqlE0r8kPVToWOqDpCWSnpX0tKSiGL1TUgdJ0yQ9L2lhOt1z3Ry72PsgJB0OrAbuiIh9Ch1P\nvknaFdg1IuZJagfMBb4aEc8VOLS8kSSgTUSsltQCeAL4XkT8s8Ch5Z2kC4BSYKeIOL7Q8eSbpCVA\naUQUzYNykm4H/hERN0lqCbSOiJV1ceyir0FExCySuSiKQkS8GRHz0vcfAAvJmO+7KUnnRVmdLrZI\nX03+l5GkbsBxwE2FjsXyQ1J74HDgZoCI+KSukgM4QRQ1ST1JJmuaXdhI8i9tankaeAf4S0Q0+XMm\nmazrx8D6QgdSjwL4s6S5ksYWOph60AtYBtyaNiXeJKlNXR3cCaJISWoL3EcyW9/7hY4n3yJiXUTs\nTzK/+YGSmnRzoqTjgXciYm6hY6lnh0bEAOBY4Ly0Cbkpaw4MAG6IiP7Ah8D4ujq4E0QRStvh7wOm\nRMT/K3Q89Smtfj8ODC10LHk2EBiWtslPBY6UNLmwIeVfRLyR/vsOcD9wYGEjyrtyoDynRjyNJGHU\nCSeIIpN22N4MLIyIqwsdT32Q1EVSh/T9jsCXgecLG1V+RcRPIqJbRPQERgKPRcSYAoeVV5LapDde\nkDazDAGa9N2JEfEW8LqkvdKio4A6u+GkeV0dqLGSdDdwBNBZUjnwi4i4ubBR5dVA4FTg2bRNHuCn\nEfFIAWPKt12B2yWVkPwoujciiuK2zyLzGeD+5DcQzYG7IuLRwoZUL74LTEnvYHoZOLOuDlz0t7ma\nmVk2NzGZmVkmJwgzM8vkBGFmZpmcIMzMLJMThJmZZXKCMNsCSevS0UErX3X2pKqknsUykrA1PkX/\nHIRZLaxNh+kwKyquQZhto3Tugd+m8w88JWmPtLynpMckzZf0V0k90vLPSLo/nZfiGUmHpIcqkXRj\nOlfFn9OnvZE0Lp23Y76kqQU6TStiThBmW7ZjlSamk3PWrYqIfYHrSUZPBbgOuD0i+gFTgGvT8muB\nv0fEfiTj5SxIy/cEJkZEX2Al8PW0fDzQPz3OOfk6ObPq+Elqsy2QtDoi2maULwGOjIiX0wEQ34qI\nTpKWk0zK9Gla/mZEdJa0DOgWER/nHKMnyfDje6bLFwEtIuI/JD1KMpnVn4A/5cxpYVYvXIMw2z5R\nzfut8XHO+3Vs7Bs8DphIUtuYI8l9hlavnCDMts/JOf8+mb7/P5IRVAFGA/9I3/8VOBc2TGDUvrqD\nSmoGdI+Ix4GLgPbAZrUYs3zyLxKzLdsxZ+RbgEcjovJW146S5pPUAkalZd8lmeHrRySzfVWOrvk9\nYJKks0hqCucCb1bzmSXA5DSJCLi2LqeSNKsN90GYbaO0D6I0IpYXOhazfHATk5mZZXINwszMMrkG\nYWZmmZwgzMwskxOEmZllcoIwM7NMThBmZpbp/wN7wjSEH/q5awAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5wWdd3/8debgwIioIBZnEtTEVhZ\nVtDEA57Cc54lzMSUsjxWlr80Nc3uyjS1vL0l85TI4faUGmqClHlbykEOCiqkKAuIgLigSLr6+f0x\ns+vFsocL2GuX3Xk/H4/rsTPf+c7MZy4u5nPN9zvzvRQRmJlZdrVo7ADMzKxxORGYmWWcE4GZWcY5\nEZiZZZwTgZlZxjkRmJllnBOBbUBSS0nvS+pZn3Ubk6RdJBXkPumq25b0V0kjCxGHpJ9K+p/NXd+s\nJk4ETVx6Iq54fSrpw5z5ak9ItYmITyKifUS8VZ91t1aSJku6opryEyUtkdRyU7YXEYdHxNh6iOtQ\nSYuqbPuaiPjOlm7brCongiYuPRG3j4j2wFvAMTllG52QJLVq+Ci3ancD36im/BvAvRHxSQPHkzn+\nTDY+J4JmTtLPJU2QNE7SWuB0SftK+pek9yQtk3SzpNZp/VaSQlLvdP7edPnjktZK+qekPptaN11+\nhKTXJJVJ+p2k/5N0Zg1x5xPjtyUtlLRa0s0567aU9FtJqyS9Dgyv5S16ENhZ0ldy1u8MHAnck84f\nK2mWpDWS3pL001re72crjqmuOCSdLWl++l79W9LZaXlH4FGgZ87V3U7pv+VdOesfL+nl9D16WtJu\nOctKJX1f0tz0/R4nadsaYt5V0lRJ70paKelPaQwVy3tJeljSinT5TTnLvi3plfQYXpJUVPVzkda7\nV9JV6fShkhZJ+omkt4E/SOosaVK6j9WSHpXULfffRNJd6WdhtaQH0vJXJB2RU2/bdHn/mv6NbGNO\nBNlwPHAf0BGYAJQDFwJdgP1ITlDfrmX9rwM/BXYkueq4ZlPrStoJmAhcku73DWBwLdvJJ8YjgUHA\nQJIEd2hafi5wOFAE7A2cUtNOIuID4H7gjJzi04A5EfFyOv8+MBLoBBwDXCjp6Fpir1BXHMuBo4AO\nwDnA7yQNiIiydD9v5VzdvZO7oqQ9gD8B5wNdgcnAIxXJMnUKcBjwRZL3qborHwABPwd2Bvqm9X+a\n7qcV8BdgIdAb6EHy74ikEcDl6XvTATgBeDeP9wWgO9Ae6Al8l+Rc9Id0vhfwMXBTTv37gG3S+HbK\nWXYPcHpOvaOBRRExN884DCAi/GomL2ARcGiVsp8DT9ex3g+B/02nWwEB9E7n7wX+J6fuscBLm1H3\nLOAfOcsELAPOzPPYqotxn5zlDwI/TKefAc7OWXZk8lGvcdsHkZzAtk3nnwfOr6X+74Hr0uldcrcN\nPFtxTJsRx2PA99LpQ0lOaFX/Le9Kp38G3JezrAXwNjA0nS8FTstZfgPw+zzf65OAaen0/ul2W1ZT\nb0pFvFXKN/hc5Hw2rso5tvXANrXEUAKsSKd7kHwx6FhNvR7AGmC7dP5h4PuF+P/VnF++IsiGxbkz\nknaX9BdJb0taA1xN8s27Jm/nTK8j+Sa3qXW/kBtHJP9rS2vaSJ4x5rUv4M1a4gX4O8nJ5BhJXya5\nwhiXE8u+kv6WNluUAWdXE0t1ao1D0tGSnk+bZN4juXrIZ7sV267cXkR8SvJ+dsupk9e/m6SdJU1U\n0jm+BrgrJ44eJAmpur6SHsC/84y3quUR8VFODO0l3Z42va0Bnq4Sw8pIrpQ2EBGLgReAEyTtSPIe\n3reZMWWWE0E2VL1l8TbgJWCXiOgAXEHyDb2QlpE0BwAgSWx40qpqS2JcRnLyqFDr7a1pUrqHpHno\nG8CkiFiZU2U88ADQIyI6ArfnGUuNcUhqS9Ik9V/A5yKiE/DXnO3WdZvpUpImlIrttSB5f5fkEVdV\nvwL+A/RP3+szc+JYDPRS9XdPLQa+VLUwIsrT7bXLKd65arUq85cAfYDBaQwHV9lPF0kdaoj/bpLm\noVOBZyLi7RrqWQ2cCLJpe6AM+CBta66tf6C+PAYUSzombXe+kKRtuxAxTgQuktQt7fj9cR7r3EPS\nD3EWyYmlaizvRsR6SfuQ9CFsaRzbkrR5rwA+SfscDslZvpzk5Ld9Lds+VtJBab/AJcBakmatTbU9\n8AFQJqkHSTNchX8Cq4BfSGonqa2k/dJltwM/kjRQiV3T9QFmAyOVdJgfBQzNI4Z1wOr0vaq8pTf9\n1j8ZuEVSJ0mtJR2Qs+6DwBDgPNIOfts0TgTZ9APgmyQnjttIOpALKiKWk3xju4HkxPIl4EWSb471\nHeOtJO3Xc4FpJN+864pvIUkTw7YknaO5zgX+S8ldVz8h7Szdkjgi4j3gYuAhkv6Jk0iSZcXyl0iu\nQhaldwXtVCXel0nen1tJkslw4NiI+DjP2HJdSdJxXwY8ku63Yj/lJB2we5B8M38rjZWIGEdyNTGB\npGntQWCHdNULSG5SeA84Od1ubW4guZlhFfAc8HiV5RUdwq+RJMnzc2L8gKRvoGf61zaR0g4WswaV\nNjUsBU6KiH80djzWtEm6GugZEWc2dixNka8IrMFIGp5e2m9LcnvixyTfws02W9qUNAoY09ixNFVO\nBNaQhgKvkzRlfBU4PiJqahoyq5Okc0maq/4cEc81djxNlZuGzMwyzlcEZmYZ1+QGe+rSpUv07t27\nscMwM2tSZsyYsTIiqr1lu8klgt69ezN9+vTGDsPMrEmRVOMT9m4aMjPLOCcCM7OMcyIwM8s4JwIz\ns4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OM\ncyIwM8u4giUCSXdIekfSSzUsl6SbJS2UNEdScaFiyaKxY6F3b2jRIvk7dmxjR1R4PmYfc3NV8GOO\niIK8gAOAYuClGpYfCTwOCNgHeD6f7Q4aNCisdvfeG9GuXQR89mrXLilvrnzMPubmqr6OGZgeNZ2v\na1pQHy+gdy2J4DZgRM78q8Dn69rm5iSCe++N6NUrQkr+NucPTURyjLkfmopXr16NHVnh+Jh9zM1V\nfR1zbYmgMfsIugGLc+ZL07KNSBotabqk6StWrNiknYwdC6NHw5tvJm/fm28m8835cvKttzatvDnw\nMddd3hz4mOsu3xxNorM4IsZERElElHTtWu1vL9fosstg3boNy9atS8qbq549N628OfAx113eHPiY\n6y7fHI2ZCJYAPXLmu6dl9Wpr/AYRAZ9+CuXl8NFHsH49fPghfPABrF0LZWXw3nvw7ruwciWsWAHL\nl8OyZbBkCZSWJvEvWgRvvAH//jcsWACvvgqvvALf/S60abPhPtu0gYsuStZfvjzZ9po1yX4//jiJ\nqSm79lpo127DsnbtkvLmysec8DFvOUUBzwCSegOPRUS/apYdBZxH0mk8BLg5IgbXtc2SkpKYPn16\n3jH07p00B1W1zTbQr19yQv70089OzlWn65rfnGVbq5YtoXVraNXqs7+50431N9+6kybBb3+bJLtu\n3eBHP4ITTvjs+KSGmW6o/QBMmABXXgmLFyffEK+6CkaMoFkbN+6zY+7RIznm005LluWezhpquiH2\n89BD8OtfJ5/tXr2SJDByJJtE0oyIKKl2WaESgaRxwEFAF2A5cCXQGiAi/keSgN8Dw4F1wKiIqPMM\nv6mJoKKPILd5qEUL6N8/+RC1aJH8x2rRYuPpuuYLsay+txORXHmUlyff/Lfkb31sI/dvU78KMWto\nt94K3/nO5q1bWyJotSVB1SYiav1ekvZif69Q+69QkTUvuyxpTunZc/OyqdW/Tz8tTDL65JNk+831\n22Ft+8yShrzy2lquAr/yFQqiYIlgazJypE/8W6MWLWDbbZOXmTWeJnHXkJmZFY4TgZlZxjkRmJll\nnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwT\ngZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZ\nWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcYVNBFIGi7pVUkLJV1azfKekqZKelHS\nHElHFjIeMzPbWMESgaSWwC3AEUBfYISkvlWqXQ5MjIiBwGnAfxcqHjMzq14hrwgGAwsj4vWI+AgY\nDxxXpU4AHdLpjsDSAsZjZmbVKGQi6AYszpkvTctyXQWcLqkUmAScX92GJI2WNF3S9BUrVhQiVjOz\nzGrszuIRwF0R0R04EviTpI1iiogxEVESESVdu3Zt8CDNzJqzQiaCJUCPnPnuaVmubwETASLin0Ab\noEsBYzIzsyoKmQimAbtK6iNpG5LO4Eeq1HkLOARA0h4kicBtP2ZmDahgiSAiyoHzgCeB+SR3B70s\n6WpJx6bVfgCcI2k2MA44MyKiUDGZmdnGWhVy4xExiaQTOLfsipzpecB+hYzBzMxq19idxWZm1sic\nCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjM\nzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwy\nzonAzCzj8koEkh6UdJQkJw4zs2Ym3xP7fwNfBxZI+qWk3QoYk5mZNaC8EkFETI6IkUAxsAiYLOk5\nSaMktS5kgGZmVlh5N/VI6gycCZwNvAjcRJIYnipIZGZm1iBa5VNJ0kPAbsCfgGMiYlm6aIKk6YUK\nzszMCi+vRADcHBFTq1sQESX1GI+ZmTWwfJuG+krqVDEjaQdJ3y1QTGZm1oDyTQTnRMR7FTMRsRo4\np66VJA2X9KqkhZIuraHOKZLmSXpZ0n15xmNmZvUk36ahlpIUEQEgqSWwTW0rpHVuAQ4DSoFpkh6J\niHk5dXYF/h+wX0SslrTT5hyEmZltvnyvCJ4g6Rg+RNIhwLi0rDaDgYUR8XpEfASMB46rUucc4Jb0\nCoOIeCf/0M3MrD7ke0XwY+DbwLnp/FPA7XWs0w1YnDNfCgypUufLAJL+D2gJXBURGyUYSaOB0QA9\ne/bMM2QzM8tHXokgIj4Fbk1f9b3/XYGDgO7AM5L65/ZHpPsfA4wBKCkpiXqOwcws0/J9jmBX4L+A\nvkCbivKI+GItqy0BeuTMd0/LcpUCz0fEx8Abkl4jSQzT8onLzMy2XL59BHeSXA2UA8OAe4B761hn\nGrCrpD6StgFOAx6pUudhkqsBJHUhaSp6Pc+YzMysHuSbCNpGxBRAEfFmRFwFHFXbChFRDpwHPAnM\nByZGxMuSrpZ0bFrtSWCVpHnAVOCSiFi1OQdiZmabJ9/O4v+kQ1AvkHQeSRNP+7pWiohJwKQqZVfk\nTAfw/fRlZmaNIN8rgguBdsAFwCDgdOCbhQrKzMwaTp1XBOmDYadGxA+B94FRBY/KzMwaTJ1XBBHx\nCTC0AWIxM7NGkG8fwYuSHgH+F/igojAiHixIVGZm1mDyTQRtgFXAwTllATgRmJk1cfk+Wex+ATOz\nZirfJ4vvJLkC2EBEnFXvEZmZWYPKt2nosZzpNsDxwNL6D8fMzBpavk1DD+TOSxoHPFuQiMzMrEHl\n+0BZVbsC/hEZM7NmIN8+grVs2EfwNslvFJiZWROXb9PQ9oUOxMzMGkdeTUOSjpfUMWe+k6SvFS4s\nMzNrKPn2EVwZEWUVM+kviF1ZmJDMzKwh5ZsIqquX762nZma2Fcs3EUyXdIOkL6WvG4AZhQzMzMwa\nRr6J4HzgI2ACMB5YD3yvUEGZmVnDyfeuoQ+ASwsci5mZNYJ87xp6SlKnnPkdJD1ZuLDMzKyh5Ns0\n1CW9UwiAiFiNnyw2M2sW8k0En0rqWTEjqTfVjEZqZmZNT763gF4GPCvp74CA/YHRBYvKzMwaTL6d\nxU9IKiE5+b8IPAx8WMjAzMysYeQ76NzZwIVAd2AWsA/wTzb86UozM2uC8u0juBDYG3gzIoYBA4H3\nal/FzMyagnwTwfqIWA8gaduIeAXYrXBhmZlZQ8m3s7g0fY7gYeApSauBNwsXlpmZNZR8O4uPTyev\nkjQV6Ag8UbCozMyswWzyCKIR8fdCBGJmZo1jc3+z2MzMmgknAjOzjCtoIpA0XNKrkhZKqnH0Ukkn\nSor0oTUzM2tABUsEkloCtwBHAH2BEZL6VlNve5LnFJ4vVCxmZlazQl4RDAYWRsTrEfERyQ/aHFdN\nvWuAX5H82I2ZmTWwQiaCbsDinPnStKySpGKgR0T8pbYNSRotabqk6StWrKj/SM3MMqzROosltQBu\nAH5QV92IGBMRJRFR0rVr18IHZ2aWIYVMBEuAHjnz3dOyCtsD/YC/SVpEMpDdI+4wNjNrWIVMBNOA\nXSX1kbQNcBrwSMXCiCiLiC4R0TsiegP/Ao6NiOkFjMnMzKooWCKIiHLgPOBJYD4wMSJelnS1pGML\ntV8zM9s0mzzExKaIiEnApCplV9RQ96BCxmJmZtXzk8VmZhnnRGBmlnFOBGZmGedEYGaWcU4EZmYZ\n50RgZpZxTgRmZhnnRGBmlnFOBGZmGedEYGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGZmGedE\nYGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGZmGedEYGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBm\nlnFOBGZmGedEYGaWcU4EZmYZ50RgZpZxBU0EkoZLelXSQkmXVrP8+5LmSZojaYqkXoWMx8zMNlaw\nRCCpJXALcATQFxghqW+Vai8CJRExALgf+HWh4jEzs+oV8opgMLAwIl6PiI+A8cBxuRUiYmpErEtn\n/wV0L2A8ZmZWjUImgm7A4pz50rSsJt8CHq9ugaTRkqZLmr5ixYp6DNHMzLaKzmJJpwMlwHXVLY+I\nMRFREhElXbt2bdjgzMyauVYF3PYSoEfOfPe0bAOSDgUuAw6MiP9szo4+/vhjSktLWb9+/WYFas1T\nmzZt6N69O61bt27sUMy2aoVMBNOAXSX1IUkApwFfz60gaSBwGzA8It7Z3B2Vlpay/fbb07t3byRt\nSczWTEQEq1atorS0lD59+jR2OGZbtYI1DUVEOXAe8CQwH5gYES9LulrSsWm164D2wP9KmiXpkc3Z\n1/r16+ncubOTgFWSROfOnX2VaJaHQl4REBGTgElVyq7ImT60vvblJGBV+TNhlp+torPYzMwaTyYT\nwdix0Ls3tGiR/B07dsu2t2rVKvbaay/22msvdt55Z7p161Y5/9FHH+W1jVGjRvHqq6/WWueWW25h\n7JYGa2ZWRUGbhrZGY8fC6NGwLn2M7c03k3mAkSM3b5udO3dm1qxZAFx11VW0b9+eH/7whxvUiQgi\nghYtqs+9d955Z537+d73vrd5ATai8vJyWrXK3MfMrEnJ3BXBZZd9lgQqrFuXlNe3hQsX0rdvX0aO\nHMmee+7JsmXLGD16NCUlJey5555cffXVlXWHDh3KrFmzKC8vp1OnTlx66aUUFRWx77778s47yQ1V\nl19+OTfeeGNl/UsvvZTBgwez22678dxzzwHwwQcfcOKJJ9K3b19OOukkSkpKKpNUriuvvJK9996b\nfv368Z3vfIeIAOC1117j4IMPpqioiOLiYhYtWgTAL37xC/r3709RURGXpW9WRcwAb7/9NrvssgsA\nt99+O1/72tcYNmwYX/3qV1mzZg0HH3wwxcXFDBgwgMcee6wyjjvvvJMBAwZQVFTEqFGjKCsr44tf\n/CLl5eUArF69eoN5M6t/mUsEb721aeVb6pVXXuHiiy9m3rx5dOvWjV/+8pdMnz6d2bNn89RTTzFv\n3ryN1ikrK+PAAw9k9uzZ7Lvvvtxxxx3VbjsieOGFF7juuusqk8rvfvc7dt55Z+bNm8dPf/pTXnzx\nxWrXvfDCC5k2bRpz586lrKyMJ554AoARI0Zw8cUXM3v2bJ577jl22mknHn30UR5//HFeeOEFZs+e\nzQ9+8IM6j/vFF1/kwQcfZMqUKbRt25aHH36YmTNnMnnyZC6++GIAZs+eza9+9Sv+9re/MXv2bK6/\n/no6duzIfvvtVxnPuHHjOPnkk31VYVZAmUsEPXtuWvmW+tKXvkRJSUnl/Lhx4yguLqa4uJj58+dX\nmwjatm3LEUccAcCgQYMqv5VXdcIJJ2xU59lnn+W0004DoKioiD333LPadadMmcLgwYMpKiri73//\nOy+//DKrV69m5cqVHHPMMUDyQFa7du2YPHkyZ511Fm3btgVgxx13rPO4Dz/8cHbYYQcgSViXXnop\nAwYM4PDDD2fx4sWsXLmSp59+mlNPPbVyexV/zz777MqmsjvvvJNRo0bVuT8z23yZSwTXXgvt2m1Y\n1q5dUl4I2223XeX0ggULuOmmm3j66aeZM2cOw4cPr/Y+92222aZyumXLljU2i2y77bZ11qnOunXr\nOO+883jooYeYM2cOZ5111mbdb9+qVSs+/fRTgI3Wzz3ue+65h7KyMmbOnMmsWbPo0qVLrfs78MAD\nee2115g6dSqtW7dm99133+TYzCx/mUsEI0fCmDHQqxdIyd8xYza/o3hTrFmzhu23354OHTqwbNky\nnnzyyXrfx3777cfEiRMBmDt3brVXHB9++CEtWrSgS5curF27lgceeACAHXbYga5du/Loo48Cycl9\n3bp1HHbYYdxxxx18+OGHALz77rsA9O7dmxkzZgBw//331xhTWVkZO+20E61ateKpp55iyZJkpJGD\nDz6YCRMmVG6v4i/A6aefzsiRI301YNYAMpcIIDnpL1oEn36a/G2IJABQXFxM37592X333TnjjDPY\nb7/96n0f559/PkuWLKFv37787Gc/o2/fvnTs2HGDOp07d+ab3/wmffv25YgjjmDIkCGVy8aOHcv1\n11/PgAEDGDp0KCtWrODoo49m+PDhlJSUsNdee/Hb3/4WgEsuuYSbbrqJ4uJiVq9eXWNM3/jGN3ju\nuefo378/48ePZ9dddwWSpqsf/ehHHHDAAey1115ccsklleuMHDmSsrIyTj311Pp8e8ysGqq4W6Sp\nKCkpienTp29QNn/+fPbYY49GimjrUl5eTnl5OW3atGHBggUcfvjhLFiwoMl1to4fP54nn3wyr9tq\na+PPhllC0oyIKKluWdM6O1id3n//fQ455BDKy8uJCG677bYmlwTOPfdcJk+eXHnnkJkVVtM6Q1id\nOnXqVNlu31TdeuutjR2CWaZkso/AzMw+40RgZpZxTgRmZhnnRGBmlnFOBPVg2LBhGz0cduONN3Lu\nuefWul779u0BWLp0KSeddFK1dQ466CCq3i5b1Y033si6nJH0jjzySN577718QjczcyKoDyNGjGD8\n+PEblI0fP54RI0bktf4XvvCFWp/MrUvVRDBp0iQ6deq02dtraBFROVSFmTW8ZpcILroIDjqofl8X\nXVT7Pk866ST+8pe/VP4IzaJFi1i6dCn7779/5X39xcXF9O/fnz//+c8brb9o0SL69esHJMM/nHba\naeyxxx4cf/zxlcM6QHJ/fcUQ1ldeeSUAN998M0uXLmXYsGEMGzYMSIZ+WLlyJQA33HAD/fr1o1+/\nfpVDWC9atIg99tiDc845hz333JPDDz98g/1UePTRRxkyZAgDBw7k0EMPZfny5UDyrMKoUaPo378/\nAwYMqByi4oknnqC4uJiioiIOOeQQIPl9ht/85jeV2+zXrx+LFi1i0aJF7Lbbbpxxxhn069ePxYsX\nV3t8ANOmTeMrX/kKRUVFDB48mLVr13LAAQdsMLz20KFDmT17du3/UGZWLT9HUA923HFHBg8ezOOP\nP85xxx3H+PHjOeWUU5BEmzZteOihh+jQoQMrV65kn3324dhjj63x93RvvfVW2rVrx/z585kzZw7F\nxcWVy6699lp23HFHPvnkEw455BDmzJnDBRdcwA033MDUqVPp0qXLBtuaMWMGd955J88//zwRwZAh\nQzjwwAPZYYcdWLBgAePGjeMPf/gDp5xyCg888ACnn376BusPHTqUf/3rX0ji9ttv59e//jXXX389\n11xzDR07dmTu3LlA8psBK1as4JxzzuGZZ56hT58+G4wbVJMFCxZw9913s88++9R4fLvvvjunnnoq\nEyZMYO+992bNmjW0bduWb33rW9x1113ceOONvPbaa6xfv56ioqJN+nczs0SzSwTpl94GV9E8VJEI\n/vjHPwJJs8dPfvITnnnmGVq0aMGSJUtYvnw5O++8c7XbeeaZZ7jgggsAGDBgAAMGDKhcNnHiRMaM\nGUN5eTnLli1j3rx5Gyyv6tlnn+X444+vHAn0hBNO4B//+AfHHnssffr0Ya+99gJqHuq6tLSUU089\nlWXLlvHRRx/Rp08fACZPnrxBU9gOO+zAo48+ygEHHFBZJ5+hqnv16lWZBGo6Pkl8/vOfZ++99wag\nQ4cOAJx88slcc801XHfdddxxxx2ceeaZde7PzKrX7JqGGstxxx3HlClTmDlzJuvWrWPQoEFAMojb\nihUrmDFjBrNmzeJzn/vcZg35/MYbb/Cb3/yGKVOmMGfOHI466qjN2k6FiiGsoeZhrM8//3zOO+88\n5s6dy2233bbFQ1XDhsNV5w5VvanH165dOw477DD+/Oc/M3HiREY21MiBZs2QE0E9ad++PcOGDeOs\ns87aoJO4Ygjm1q1bM3XqVN58881at3PAAQdw3333AfDSSy8xZ84cIBnCervttqNjx44sX76cxx9/\nvHKd7bffnrVr1260rf3335+HH36YdevW8cEHH/DQQw+x//77531MZWVldOvWDYC77767svywww7j\nlltuqZxfvXo1++yzD8888wxvvPEGsOFQ1TNnzgRg5syZlcurqun4dtttN5YtW8a0adMAWLt2bWXS\nOvvss7ngggvYe++9K38Ex8w2nRNBPRoxYgSzZ8/eIBGMHDmS6dOn079/f+655546f2Tl3HPP5f33\n32ePPfbgiiuuqLyyKCoqYuDAgey+++58/etf32AI69GjRzN8+PDKzuIKxcXFnHnmmQwePJghQ4Zw\n9tlnM3DgwLyP56qrruLkk+lMjh4AAAaGSURBVE9m0KBBG/Q/XH755axevZp+/fpRVFTE1KlT6dq1\nK2PGjOGEE06gqKiocvjoE088kXfffZc999yT3//+93z5y1+udl81Hd8222zDhAkTOP/88ykqKuKw\nww6rvFIYNGgQHTp08G8WmG0hD0NtTdbSpUs56KCDeOWVV2jRovrvNP5smCVqG4baVwTWJN1zzz0M\nGTKEa6+9tsYkYGb5aXZ3DVk2nHHGGZxxxhmNHYZZs9Bsvko1tSYuKzx/Jszy0ywSQZs2bVi1apX/\n41uliGDVqlW0adOmsUMx2+o1i6ah7t27U1payooVKxo7FNuKtGnThu7duzd2GGZbvWaRCFq3bl35\nRKuZmW2agjYNSRou6VVJCyVdWs3ybSVNSJc/L6l3IeMxM7ONFSwRSGoJ3AIcAfQFRkjqW6Xat4DV\nEbEL8FvgV4WKx8zMqlfIK4LBwMKIeD0iPgLGA8dVqXMcUDF2wf3AIappWE4zMyuIQvYRdAMW58yX\nAkNqqhMR5ZLKgM7AytxKkkYDo9PZ9yW9upkxdam67QzwMWeDjzkbtuSYe9W0oEl0FkfEGGDMlm5H\n0vSaHrFurnzM2eBjzoZCHXMhm4aWAD1y5runZdXWkdQK6AisKmBMZmZWRSETwTRgV0l9JG0DnAY8\nUqXOI8A30+mTgKfDT4WZmTWogjUNpW3+5wFPAi2BOyLiZUlXA9Mj4hHgj8CfJC0E3iVJFoW0xc1L\nTZCPORt8zNlQkGNucsNQm5lZ/WoWYw2ZmdnmcyIwM8u4TCQCSXdIekfSS40dS0OR1EPSVEnzJL0s\n6cLGjqnQJLWR9IKk2ekx/6yxY2oIklpKelHSY40dS0OQtEjSXEmzJE2ve42mT1InSfdLekXSfEn7\n1uv2s9BHIOkA4H3gnojo19jxNARJnwc+HxEzJW0PzAC+FhHzGjm0gkmfSt8uIt6X1Bp4FrgwIv7V\nyKEVlKTvAyVAh4g4urHjKTRJi4CSiMjMw2SS7gb+ERG3p3dhtouI9+pr+5m4IoiIZ0juSsqMiFgW\nETPT6bXAfJInuZutSLyfzrZOX836m46k7sBRwO2NHYsVhqSOwAEkd1kSER/VZxKAjCSCrEtHdR0I\nPN+4kRRe2kwyC3gHeCoimvsx3wj8CPi0sQNpQAH8VdKMdPiZ5q4PsAK4M20CvF3SdvW5AyeCZk5S\ne+AB4KKIWNPY8RRaRHwSEXuRPMk+WFKzbQqUdDTwTkTMaOxYGtjQiCgmGdn4e2nTb3PWCigGbo2I\ngcAHwEbD+m8JJ4JmLG0nfwAYGxEPNnY8DSm9dJ4KDG/sWApoP+DYtM18PHCwpHsbN6TCi4gl6d93\ngIdIRjpuzkqB0pyr2/tJEkO9cSJoptKO0z8C8yPihsaOpyFI6iqpUzrdFjgMeKVxoyqciPh/EdE9\nInqTPJX/dESc3shhFZSk7dKbH0ibRw4HmvXdgBHxNrBY0m5p0SFAvd700SRGH91SksYBBwFdJJUC\nV0bEHxs3qoLbD/gGMDdtMwf4SURMasSYCu3zwN3pjyK1ACZGRCZuqcyQzwEPpT9b0gq4LyKeaNyQ\nGsT5wNj0jqHXgVH1ufFM3D5qZmY1c9OQmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmKUkfZKO\naFnxqrenNyX1ztLot9a0ZOI5ArM8fZgOT2GWKb4iMKtDOv79r9Mx8F+QtEta3lvS05LmSJoiqWda\n/jlJD6W/izBb0lfSTbWU9If0txL+mj79jKQL0t+NmCNpfCMdpmWYE4HZZ9pWaRo6NWdZWUT0B35P\nMuInwO+AuyNiADAWuDktvxn4e0QUkYwJ83JavitwS0TsCbwHnJiWXwoMTLfznUIdnFlN/GSxWUrS\n+xHRvpryRcDBEfF6OpDf2xHRWdJKkh//+TgtXxYRXSStALpHxH9yttGbZFjsXdP5HwOtI+Lnkp4g\n+eGkh4GHc35TwaxB+IrALD9Rw/Sm+E/O9Cd81kd3FHALydXDNEnuu7MG5URglp9Tc/7+M51+jmTU\nT4CRwD/S6SnAuVD5Qzkda9qopBZAj4iYCvwY6AhsdFViVkj+5mH2mbY5I7UCPBERFbeQ7iBpDsm3\n+hFp2fkkvxp1CckvSFWMCHkhMEbSt0i++Z8LLKthny2Be9NkIeDm+v4ZQrO6uI/ArA5Z/LF0yxY3\nDZmZZZyvCMzMMs5XBGZmGedEYGaWcU4EZmYZ50RgZpZxTgRmZhn3/wGQiGhIXoDCyQAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5gU9Z3v8feHmwOCEAFvgA7HGBEv\nXJxDlqhEY+LiDY6KChI1GkPW9ZqoG2KMG02MyZpHXQ3HDTExZsVBolFZV9ckSrys8TIoIhddOWTU\nQcQJIIrcHP2eP6oYm6HnBtPdzNTn9Tz90PWr6l99q4enP12/qq5SRGBmZtnVqdQFmJlZaTkIzMwy\nzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwE1mYkdZa0VtLebblsKUn6rKSCnGPdsG9Jf5A0uRB1SPq+\npH/b1tdbx+YgyLD0g3jz4xNJ63Om834gNSUiPo6InhHxZlsuu6OS9CdJV+dpP0XSMkmdW9NfRBwT\nETPaoK4vS6pu0PcPI+IftrfvZtYZki5rrpa0/WlJX8uZHiLpXkkrJa2R9LKkSyX5M6oI/CZnWPpB\n3DMiegJvAifmtG31gSSpS/Gr3KHdCZyZp/1M4K6I+LjI9ZTS2cAq4KzWvlDSfsCzwFLgoIjoDUwE\nRgM92rJIy89BYI2S9CNJ90iqlPQB8FVJoyU9K+k9Scsl3SKpa7p8l/RbYXk6fVc6/xFJH0j6i6TB\nrV02nX+spP9Jvy3eKum/c79RNqi7JTV+U9ISSasl3ZLz2s6Sbkq/mS4FxjbxFv0e2EPSF3Je3xc4\nDvhtOj1O0jxJ70t6U9L3m3i/678lN1eHpPMkLU7fq/8n6by0vTfwH8DeOXt3u6V/y9/kvP4kSQvT\n9+hxSfvnzKuR9G1Jr6Tvd6WknZqouxdwMvCPwFBJw5t4z/L5IfBERPxTRCwHiIjFEXF6RKxtZV+2\nDRwE1pyTgLuB3sA9QB1wCdAPOIzkA+qbTbz+DOD7wK4kex0/bO2yknYDZgFXpOv9KzCqiX5aUuNx\nwKHACJKA+3Lafj5wDDAM+N/AaY2tJCI+BO5ly2/BE4H5EbEwnV4LTAb6ACcCl0g6oYnaN2uujhXA\n8cAuwDeAWyUdEhFr0vW8mbN3927uCyUdAPw7cBHQH/gTMHtzWKZOA74C/C+S9ynfns9mE4DVwO/S\nvs5uwfbl+jLJ+2gl4iCw5jwdEf8REZ9ExPqIeCEinouIuohYCkwHvtjE6++NiKqI+AiYATT1bbGx\nZU8A5kXEg+m8m4C/NdZJC2u8PiLWREQ18OecdZ0G3BQRNRGxEvhJE/VCMjx0Ws435rPSts21PB4R\nC9P372VgZp5a8mmyjvRvsjQSjwOPAUe0oF9Iwmp2WttHad+9gc/nLHNzRLyTrvshmv67nQ3MjIhP\nSL40nNHKYcRdgeWtWN7amIPAmvNW7kR6UO8/Jb0j6X3gWpJv3o15J+f5OqDnNiy7V24dkVwpsaax\nTlpYY4vWBbzRRL0ATwDvAydK+hzJHkZlTi2jJf1ZUq2kNcB5eWrJp8k6JJ0g6TlJqyS9R7L30JJ+\nN/dd31/6AV4DDMhZpkV/t3RobwxJcAPcny67eSirDui61QuTto/S56uAPVtYuxWAg8Ca0/CUxV8A\nC4DPRsQuwNWAClzDcmDg5glJYssPrYa2p8blwKCc6SZPb01D6bckewJnAg9HRO7eykzgPmBQehD0\n9hbW0mgdkrqTDKVcD+weEX2AP+T029xppm8D++T014nk/V3WgroaOitd7yOS3gGWAN34dHjoTWA3\nSfUHfdO/3z58GkZ/Ak7ZhnVbG3EQWGv1AtYAH6ZjzU0dH2grDwEjJZ2YDjlcQjK2XYgaZwGXShqQ\nHvj9Tgte81uSb8DnkjMslFPLqojYIOnvSIZltreOnUg+bGuBj9NjDkfnzF8B9EsP4jbW9zhJR6bH\nBa4APgCea2Ftuc4iCdrhOY/TSfaQPkNyPOcl4CeSdpZUBkwFPgReSPu4GjhS0vWS9gCQ9DlJd0tq\nag/S2oiDwFrrMpJvex+QfPO+p9ArjIgVJB8uNwIrgX1JPlw2FqDG20jG218h+aBq9iBmRCwBnif5\ngP7PBrPPB65XctbVlSQfwttVR0S8B3yLZBhmFcnB2ody5i8g2QupTs8K2q1BvQtJ3p/bSMJkLDAu\nPV7QYpIOJxlmmpYeT3gnIt5J66oGTk/3mCaQ7MEtJRmC+iJwfERsTOv5H5JTRT8HLEqHumaRnFK6\nrjU12baRb0xj7Y2SH2q9DUyIiKdKXY9Ze+c9AmsXJI2V1Cc9O+f7JAcany9xWWYdgoPA2ovDSYYW\naoG/B07aPLRgZtvHQ0NmZhnnPQIzs4xrdxcR69evX5SXl5e6DDOzdmXu3Ll/i4i8p123uyAoLy+n\nqqqq1GWYmbUrkhr9lbyHhszMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnG\nOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzy7iCBYGkX0t6V9KCRuZL0i2Slkia\nL2lkoWqZMQPKy6FTp+TfGTMKtaYdh7fZ29xReZsLsM0RUZAHMAYYCSxoZP5xwCOAgL8DnmtJv4ce\nemi0xl13RfToEQGfPnr0SNo7Km+zt7mj8jZv+zYDVdHI52pBb14vqRx4KCIOyjPvF8CfI6IynX4N\nODIiljfVZ0VFRbTmDmXl5fBGnvvy7LMPVFe3uJt2xdv8KW9zx+Jt/lRrt1nS3IioyDevlMcIBgBv\n5UzXpG1bkTRFUpWkqtra2lat5M03W9feEXibm2/vCLzNzbd3BMXY5nZxsDgipkdERURU9O+f997L\njdp779a1dwTe5ubbOwJvc/PtHUExtrmUQbAMGJQzPTBta1PXXQc9emzZ1qNH0t5ReZsT3uaOx9uc\naOtt7tJ2XbXabOBCSTOBzwNrmjs+sC0mT07+vegiWL06Oereowf88Idwww1QVgbduyf/5j4atm3P\ndKcix+3mbf7e95Ldx733Tv7TbG7viLzN3uaOqhjbXLCDxZIqgSOBfsAK4J+BrgAR8W+SBPwcGAus\nA86JiGaPArf2YPFmjz8Os2fDhg2fPtavb9l0XV2rV7eFrl1bHyRtEUZlZdC58/bVbmYdQ1MHiwt6\n1lAhbGsQbI+6Oti4seXB0ZbT2xtCZWWw665bPvr23bqt4bwePUBqm/fPzEqvqSAo5dBQu9GlS/LY\neefir3tzCOULipaEyQcfJENiK1fCqlWwZAk8/3wyvXFj4+vt1q3poGhsXs+eDhCz9sZBsIMrZAit\nX5+Ew+aQaOyxcmVyvvKLLybT69Y1Xe+2BMguuzhAzErFQZBh3bvDgAHJozU2bEj2MnKDorEQqamB\n+fOT52vXNt5n587wmc+0PkB69y7+wfjttfn3oZ98kjw+/vjT56155L6uc+dkL27zY6edPn3e3t4f\nKz4HgbVaWRnsuWfyaI1Nm7YMkKZCZMUKWLQoef7++433KW0dID16bPuHa1t9MDf1KPZhuaZCouGj\nVPO6dvUeYSk5CKxounWD3XdPHq3x0Ufw3nv5h6zytdXUJB9+nTo1/ejSZeu2lryusce2vnZ71ykl\nAbNpU3LcZ9Om/I+WzHvvvZa9rhC6dm1dgGRxT+cf/xGOO67t+3UQ2A6va1fo3z95WOlFJCcxtEXw\nbMu8jRuTvcR2dsJjm1i/vjD9OgjMrFWkJJy7di3NmXTW9jK4c2VmZrkcBGZmGecgMDPLOAeBmVnG\nOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIz\ns4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWVcQYNA0lhJ\nr0laImlqnvl7S5oj6SVJ8yUdV8h6zMxsawULAkmdgWnAscBQYJKkoQ0WuwqYFREjgInA/y1UPWZm\nll8h9whGAUsiYmlEbAJmAuMbLBPALunz3sDbBazHzMzyKGQQDADeypmuSdty/QD4qqQa4GHgonwd\nSZoiqUpSVW1tbSFqNTPLrFIfLJ4E/CYiBgLHAf8uaauaImJ6RFREREX//v2LXqSZWUdWyCBYBgzK\nmR6YtuX6OjALICL+ApQB/QpYk5mZNVDIIHgB2E/SYEndSA4Gz26wzJvA0QCSDiAJAo/9mJkVUcGC\nICLqgAuBR4HFJGcHLZR0raRx6WKXAd+Q9DJQCXwtIqJQNZmZ2da6FLLziHiY5CBwbtvVOc8XAYcV\nsgYzM2taqQ8Wm5lZiTkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5\nCMzMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOz\njHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4woa\nBJLGSnpN0hJJUxtZ5jRJiyQtlHR3IesxM7OtdSlUx5I6A9OArwA1wAuSZkfEopxl9gO+CxwWEasl\n7VaoeszMLL9C7hGMApZExNKI2ATMBMY3WOYbwLSIWA0QEe8WsB4zM8ujkEEwAHgrZ7ombcv1OeBz\nkv5b0rOSxubrSNIUSVWSqmprawtUrplZNpX6YHEXYD/gSGAS8EtJfRouFBHTI6IiIir69+9f5BLN\nzDq2RoNA0t9LmpCnfYKkr7Sg72XAoJzpgWlbrhpgdkR8FBF/Bf6HJBjMzKxImtojuBp4Ik/7n4Fr\nW9D3C8B+kgZL6gZMBGY3WOYBkr0BJPUjGSpa2oK+zcysjTQVBDtFxFYD8hHxN2Dn5jqOiDrgQuBR\nYDEwKyIWSrpW0rh0sUeBlZIWAXOAKyJiZWs3wszMtl1Tp4/uIqlL+oFeT1JXoHtLOo+Ih4GHG7Rd\nnfM8gG+nDzMzK4Gm9gh+T3Lwtv7bv6SewL+l88zMrANoKgiuAlYAb0iaK+lF4K9AbTrPzMw6gEaH\nhtIhoamSrgE+mzYviYj1RanMzMyKotEgkHRyg6YA+kiaFxEfFLYsMzMrlqYOFp+Yp21X4BBJX4+I\nxwtUk5mZFVFTQ0Pn5GuXtA8wC/h8oYoyM7PiafUlJiLiDaBrAWoxM7MSaHUQSBoCbCxALWZmVgJN\nHSz+D5IDxLl2BfYEvlrIoszMrHiaOlj8swbTAawiCYOvAn8pVFFmZlY8TR0srr/gnKQRwBnAqSQ/\nKruv8KWZmVkxNDU09DmSewRMAv4G3AMoIo4qUm1mZlYETQ0NvQo8BZwQEUsAJH2rKFWZmVnRNHXW\n0MnAcmCOpF9KOhpQccoyM7NiaTQIIuKBiJgIDCG5V8ClwG6SbpN0TLEKNDOzwmr2dwQR8WFE3B0R\nJ5LcbvIl4DsFr8zMzIqiVT8oi4jV6Y3kjy5UQWZmVlyt/mWxmZl1LA4CM7OMcxCYmWWcg8DMLOMc\nBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzy7iCBoGk\nsZJek7RE0tQmljtFUkiqKGQ9Zma2tYIFgaTOwDTgWGAoMEnS0DzL9QIuAZ4rVC1mZta4Qu4RjAKW\nRMTSiNgEzATG51nuh8BPgQ0FrMXMzBpRyCAYALyVM12TttWTNBIYFBH/2VRHkqZIqpJUVVtb2/aV\nmpllWMkOFkvqBNwIXNbcsuntMSsioqJ///6FL87MLEMKGQTLgEE50wPTts16AQcBf5ZUDfwdMNsH\njM3MiquQQfACsJ+kwZK6AROB2ZtnRsSaiOgXEeURUQ48C4yLiKoC1mRmZg0ULAgiog64EHgUWAzM\nioiFkq6VNK5Q6zUzs9bpUsjOI+Jh4OEGbVc3suyRhazFzMzy8y+LzcwyzkFgZpZxDgIzs4xzEJiZ\nZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecg\nMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwy\nzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZV9AgkDRW0muSlkiammf+tyUtkjRf0mOS9ilkPWZm\ntrWCBYGkzsA04FhgKDBJ0tAGi70EVETEIcC9wL8Uqh4zM8uvkHsEo4AlEbE0IjYBM4HxuQtExJyI\nWJdOPgsMLGA9ZmaWRyGDYADwVs50TdrWmK8Dj+SbIWmKpCpJVbW1tW1YopmZ7RAHiyV9FagAbsg3\nPyKmR0RFRFT079+/uMWZmXVwXQrY9zJgUM70wLRtC5K+DHwP+GJEbNyWFX300UfU1NSwYcOGbSrU\nSqOsrIyBAwfStWvXUpdilmmFDIIXgP0kDSYJgInAGbkLSBoB/AIYGxHvbuuKampq6NWrF+Xl5Uja\nnpqtSCKClStXUlNTw+DBg0tdjlmmFWxoKCLqgAuBR4HFwKyIWCjpWknj0sVuAHoCv5M0T9LsbVnX\nhg0b6Nu3r0OgHZFE3759vRdntgMo5B4BEfEw8HCDtqtznn+5rdblEGh//Dcz2zHsEAeLzcysdDIZ\nBDNmQHk5dOqU/Dtjxvb1t3LlSoYPH87w4cPZY489GDBgQP30pk2bWtTHOeecw2uvvdbkMtOmTWPG\n9habY8WKFXTp0oXbb7+9vq2uro4+ffpssdztt9/OpZdeWj/9m9/8hoMOOoiDDz6YkSNHctNNN7VZ\nTWZWfAUdGtoRzZgBU6bAuvRnbG+8kUwDTJ68bX327duXefPmAfCDH/yAnj17cvnll2+xTEQQEXTq\nlD9777jjjmbXc8EFF2xbgY2YNWsWo0ePprKykvPOO69Fr3nooYf4+c9/zp/+9Cf22GMPNmzYwF13\n3dWmdZlZcWVuj+B73/s0BDZbty5pb2tLlixh6NChTJ48mQMPPJDly5czZcoUKioqOPDAA7n22mvr\nlz388MOZN29e/TfyqVOnMmzYMEaPHs277yYnVF111VXcfPPN9ctPnTqVUaNGsf/++/PMM88A8OGH\nH3LKKacwdOhQJkyYQEVFRX1INVRZWcnNN9/M0qVLWb58eYu26cc//jE33ngje+yxB5CcAtrSEDGz\nHVPmguDNN1vXvr1effVVvvWtb7Fo0SIGDBjAT37yE6qqqnj55Zf54x//yKJFi7Z6zZo1a/jiF7/I\nyy+/zOjRo/n1r3+dt++I4Pnnn+eGG26oD5Vbb72VPfbYg0WLFvH973+fl156Ke9rq6urWbVqFYce\neiinnnoqs2bNatH2LFy4kEMPPbSFW29m7UHmgmDvvVvXvr323XdfKioq6qcrKysZOXIkI0eOZPHi\nxXmDoHv37hx77LEAHHrooVRXV+ft++STT95qmaeffpqJEycCMGzYMA488MC8r505cyann346ABMn\nTqSyshJo/Ewen+Fj1nFlLgiuuw569NiyrUePpL0Qdt555/rnr7/+Ov/6r//K448/zvz58xk7dmze\n8+i7detW/7xz587U1dXl7XunnXZqdpnGVFZWcvvtt1NeXs7JJ5/Miy++yNKlS+ncuTOdOnXaor9V\nq1bRr18/AIYOHcrcuXNbtS4z27FlLggmT4bp02GffUBK/p0+fdsPFLfG+++/T69evdhll11Yvnw5\njz76aJuv47DDDqsf5nnllVfy7nEsWrSIuro6li1bRnV1NdXV1VxxxRXMnDkTgDFjxnD33XcDsG7d\nOn73u99x1FFHAfDd736Xyy+/nBUrVgCwceNGfvWrX7X5dphZ8WTurCFIPvSL8cHf0MiRIxk6dChD\nhgxhn3324bDDDmvzdVx00UWcddZZDB06tP7Ru3fvLZaprKzkpJNO2qLtlFNO4eyzz+bKK6/k1ltv\n5Zvf/CY33ngjn3zyCeeeey5f+MIXABg3bhy1tbV86UtfApIho2984xttvh1mVjyKiFLX0CoVFRVR\nVVW1RdvixYs54IADSlTRjqWuro66ujrKysp4/fXXOeaYY3j99dfp0mXHzHz/7cyKQ9LciKjIN2/H\n/HSwbbZ27VqOPvpo6urqiAh+8Ytf7LAhYGY7Bn9CdDB9+vTxwVwza5XMHSw2M7MtOQjMzDLOQWBm\nlnEOAjOzjHMQtIGjjjpqqx+H3XzzzZx//vlNvq5nz54AvP3220yYMCHvMkceeSQNT5dt6Oabb2Zd\nzpX0jjvuON57772WlN4iw4cPr79sRWN1VVdXc9BBB9VPP//884wZM4b999+fESNGcN55521Ro5nt\nOBwEbWDSpEn1v8rdbObMmUyaNKlFr99rr7249957t3n9DYPg4Ycf3uqeAttq8eLFfPzxxzz11FN8\n+OGHLXrNihUrOPXUU/npT3/Ka6+9xksvvcTYsWP54IMP2qQmM2tbHe700UsvhUauurzNhg+H9OrP\neU2YMIGrrrqKTZs20a1bN6qrq3n77bc54ogjWLt2LePHj2f16tV89NFH/OhHP2L8+PFbvL66upoT\nTjiBBQsWsH79es455xxefvllhgwZwvr16+uXO//883nhhRdYv349EyZM4JprruGWW27h7bff5qij\njqJfv37MmTOH8vJyqqqq6NevHzfeeGP91UvPO+88Lr30Uqqrqzn22GM5/PDDeeaZZxgwYAAPPvgg\n3bt332rbKisrOfPMM1m8eDEPPvggZ5xxRrPv17Rp0zj77LMZPXr0Fu+Rme2YvEfQBnbddVdGjRrF\nI488AiR7A6eddhqSKCsr4/777+fFF19kzpw5XHbZZTT1a+7bbruNHj16sHjxYq655potfhNw3XXX\nUVVVxfz583niiSeYP38+F198MXvttRdz5sxhzpw5W/Q1d+5c7rjjDp577jmeffZZfvnLX9Zflvr1\n11/nggsuYOHChfTp04f77rsvbz333HMPEydOZNKkSfVXKG3OggULfKlqs3akw+0RNPXNvZA2Dw+N\nHz+emTNn1l+ILSK48sorefLJJ+nUqRPLli1jxYoV9Td2aejJJ5/k4osvBuCQQw7hkEMOqZ83a9Ys\npk+fTl1dHcuXL2fRokVbzG/o6aef5qSTTqq/AurJJ5/MU089xbhx4xg8eDDDhw8HGr/U9ea9ir33\n3psBAwZw7rnnsmrVKnbddde8l6X2parN2ifvEbSR8ePH89hjj/Hiiy+ybt26+m/EM2bMoLa2lrlz\n5zJv3jx23333vJeebs5f//pXfvazn/HYY48xf/58jj/++G3qZ7PNl7CGxi9jXVlZyauvvkp5eTn7\n7rsv77//fv2eQ9++fVm9enX9srmXqj7wwAP962azdsRB0EZ69uzJUUcdxbnnnrvFQeI1a9aw2267\n0bVrV+bMmcMbb7zRZD+5l4BesGAB8+fPB5JLWO+888707t2bFStW1A9DAfTq1SvvgdgjjjiCBx54\ngHXr1vHhhx9y//33c8QRR7Roez755BNmzZrFK6+8Un+p6gcffLB+eOjII4/krrvuqh/muvPOO+sv\nVX3hhRdy55138txzz9X39/vf/77+0tVmtmPpcENDpTRp0iROOumkLc4gmjx5MieeeCIHH3wwFRUV\nDBkypMk+zj//fM455xwOOOAADjjggPo9i2HDhjFixAiGDBnCoEGDtriE9ZQpUxg7dmz9sYLNRo4c\nyde+9jVGjRoFJAeLR4wY0egdz3I99dRTDBgwgL322qu+bcyYMSxatKj+3suvvvoqw4YNQxIVFRVc\nf/31AOy+++7MnDmTyy+/nHfffZdOnToxZswYxo4d2/ybaGZF58tQW0n5b2dWHE1dhtpDQ2ZmGecg\nMDPLuA4TBO1tiMv8NzPbUXSIICgrK2PlypX+YGlHIoKVK1dSVlZW6lLMMq9DnDU0cOBAampqqK2t\nLXUp1gplZWUMHDiw1GWYZV6HCIKuXbsyePDgUpdhZtYuFXRoSNJYSa9JWiJpap75O0m6J53/nKTy\nQtZjZmZbK1gQSOoMTAOOBYYCkyQNbbDY14HVEfFZ4Cbgp4Wqx8zM8ivkHsEoYElELI2ITcBMYHyD\nZcYDd6bP7wWOlq9cZmZWVIU8RjAAeCtnugb4fGPLRESdpDVAX+BvuQtJmgJMSSfXSnptG2vq17Dv\nDPA2Z4O3ORu2Z5v3aWxGuzhYHBHTgenb24+kqsZ+Yt1ReZuzwducDYXa5kIODS0DBuVMD0zb8i4j\nqQvQG1hZwJrMzKyBQgbBC8B+kgZL6gZMBGY3WGY2cHb6fALwePhXYWZmRVWwoaF0zP9C4FGgM/Dr\niFgo6VqgKiJmA78C/l3SEmAVSVgU0nYPL7VD3uZs8DZnQ0G2ud1dhtrMzNpWh7jWkJmZbTsHgZlZ\nxmUiCCT9WtK7khaUupZikTRI0hxJiyQtlHRJqWsqNEllkp6X9HK6zdeUuqZikNRZ0kuSHip1LcUg\nqVrSK5LmSapq/hXtn6Q+ku6V9KqkxZJGt2n/WThGIGkMsBb4bUQcVOp6ikHSnsCeEfGipF7AXOD/\nRMSiEpdWMOmv0neOiLWSugJPA5dExLMlLq2gJH0bqAB2iYgTSl1PoUmqBioiIjM/JpN0J/BURNye\nnoXZIyLea6v+M7FHEBFPkpyVlBkRsTwiXkyffwAsJvkld4cVibXpZNf00aG/6UgaCBwP3F7qWqww\nJPUGxpCcZUlEbGrLEICMBEHWpVd1HQE8V9pKCi8dJpkHvAv8MSI6+jbfDPwT8EmpCymiAP4gaW56\n+ZmObjBQC9yRDgHeLmnntlyBg6CDk9QTuA+4NCLeL3U9hRYRH0fEcJJfso+S1GGHAiWdALwbEXNL\nXUuRHR4RI0mubHxBOvTbkXUBRgK3RcQI4ENgq8v6bw8HQQeWjpPfB8yIiN+Xup5iSned5wBjS11L\nAR0GjEvHzGcCX5J0V2lLKryIWJb++y5wP8mVjjuyGqAmZ+/2XpJgaDMOgg4qPXD6K2BxRNxY6nqK\nQVJ/SX3S592BrwCvlraqwomI70bEwIgoJ/lV/uMR8dUSl1VQknZOT34gHR45BujQZwNGxDvAW5L2\nT5uOBtr0pI92cfXR7SWpEnBkg9UAAAIzSURBVDgS6CepBvjniPhVaasquMOAM4FX0jFzgCsj4uES\n1lRoewJ3pjdF6gTMiohMnFKZIbsD96e3LekC3B0R/1XakoriImBGesbQUuCctuw8E6ePmplZ4zw0\nZGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMEtJ+ji9ouXmR5v9elNSeZaufmvtSyZ+R2DWQuvT\ny1OYZYr3CMyakV7//l/Sa+A/L+mzaXu5pMclzZf0mKS90/bdJd2f3hfhZUlfSLvqLOmX6b0S/pD+\n+hlJF6f3jZgvaWaJNtMyzEFg9qnuDYaGTs+ZtyYiDgZ+TnLFT4BbgTsj4hBgBnBL2n4L8EREDCO5\nJszCtH0/YFpEHAi8B5yStk8FRqT9/EOhNs6sMf5lsVlK0tqI6JmnvRr4UkQsTS/k905E9JX0N5Kb\n/3yUti+PiH6SaoGBEbExp49yksti75dOfwfoGhE/kvRfJDdOegB4IOeeCmZF4T0Cs5aJRp63xsac\n5x/z6TG644FpJHsPL0jysTsrKgeBWcucnvPvX9Lnz5Bc9RNgMvBU+vwx4Hyov1FO78Y6ldQJGBQR\nc4DvAL2BrfZKzArJ3zzMPtU950qtAP8VEZtPIf2MpPkk3+onpW0Xkdw16gqSO0htviLkJcB0SV8n\n+eZ/PrC8kXV2Bu5Kw0LALW19G0Kz5vgYgVkzsnizdMsWDw2ZmWWc9wjMzDLOewRmZhnnIDAzyzgH\ngZlZxjkIzMwyzkFgZpZx/x+Z+CDubUbhEAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZwU9Z3/8deHQ3C4BQmEwyGRFcfI\ncExAEBQk8YfGgAdREZPV6BJZT0zyk1xG3Zi4G4NXeBBRdHGdgESXSLJ4g9eiyICAMqggIgwgjARB\nbgY++0fVzDTD3NM1zXS9n49HP6brW9XVn+qGend96zJ3R0RE4qtRqgsQEZHUUhCIiMScgkBEJOYU\nBCIiMacgEBGJOQWBiEjMKQikxsyssZntMrPuyZw2lczsZDOL5FjqsvM2sxfNbFwUdZjZr8zsT7V9\n/bHGzN40s6vC59ea2auprSg9KQhiIFwRFz8Om9nehOFyV0iVcfdD7t7S3dcnc9pjlZm9bGa3l9N+\niZltNLPGNZmfu5/r7rlJqOtbZrauzLz/zd2vq+u8y3mva83sUPhvZqeZvWtm5yX7fSQ1FAQxEK6I\nW7p7S2A98N2EtqNWSGbWpP6rPKbNAL5fTvv3gSfd/VA915Mqb4T/htoCjwKzzaxVimuSJFAQCGb2\nGzN7ysxmmtmXwJVmNsjM3jazL8xss5k9aGZNw+mbmJmbWWY4/GQ4/jkz+9LM3jKzHjWdNhx/npl9\nZGY7zOwhM/vf4q6BcuquTo0/MrM1ZrbdzB5MeG1jM7vPzLaZ2VpgZCUf0X8DncxscMLr2wPnA0+E\nw6PMbFn4a3m9mf2qks87sbuj0jrCX+Krws/qYzO7NmxvA/wN6J6wddcx/C7/M+H1F5nZyvAzmm9m\npySMKzCzW83svfDznmlmzSr5HABw98PAfwEtgZMT5ndmwvexzMzOSvy8zOw/w+9pu5k9k9A+z8wK\nw/a/mVmXqmqQ5FIQSLGLgD8DbYCngCLgZqADcCbBCupHlbz+CuBXwAkEWx3/VtNpzawjMBv4afi+\nnwADKplPdWo8H+gP9CUIuG+F7ROAc4Fs4JvApRW9ibvvBp4GfpDQfDmwwt1XhsO7gHEEv5a/C9xs\nZhdUUnuxqurYAnwHaA38C/CQmfV29x3h+6xP2LrbmvhCMzuVYIV9I3Ai8DIwtzgsQ5cC3wa+RvA5\nlbflc4Rwi/Fq4ACwIWzrBswFfk3wvU4C/jsMTAj+bR0HZAEdgQfC9kbAI0B34CTgYMI4qScKAin2\nprv/zd0Pu/ted1/s7ovcvcjd1wLTgLMref3T7p7n7geBXKBPLaa9AFjm7s+G4+4DPq9oJtWs8Xfu\nvsPd1wGvJrzXpcB97l7g7tuAeyqpF4LuoUsTfjH/IGwrrmW+u68MP7/lwKxyailPpXWE38laD8wH\nXgGGVmO+EITV3LC2g+G82wADE6a5390/C9/771T+vQ0xsy+AvcDvgCvcvfj7+UH4Xi+En8HzwHJg\nZBgSI4AJ7r7d3Q+6++vh8hW6+5zw39xO4LdU73OTJFIQSLENiQNm1svM/sfMPjOzncBdBL+8K/JZ\nwvM9BN0GNZ32q4l1eHBFxIKKZlLNGqv1XsCnldQL8BqwE/iumf0TwRbGzIRaBpnZq2EXxw7g2nJq\nKU+ldZjZBWa2yMz+Ea6Ez63mfIvnXTK/sEunAEjseqnJ9/amu7cl+MU/DxiSMO4kYGzYLfRFWOsZ\nYQ3dgM/DrZgjmFlLM3s07E7bCcyvwfJJkigIpFjZQxYfBt4HTnb31sDtgEVcw2aga/GAmRlHrrTK\nqkuNmwlWUMUqPbw1DKUnCH75fh+Yl/BrGIItgGeAbu7ehmBnanVqqbAOMzueoEvqd8BXwpXwiwnz\nreow000EK+ji+TUi+Hw3VqOuCrn7lwRdWteYWe+weQPwuLu3TXi0cPffh+M6mFnrcmb3U6AHMCD8\nDs+pS21SOwoCqUgrYAewO+xrrmz/QLL8HehnZt8N+6FvJujbjqLG2cAtZtYl7Me+rRqveYJgP8QP\nSegWSqjlH+6+z8zOIOiWqWsdzQj61QuBQ+E+hxEJ47cQrGArOnJnNjDKzIaF+wV+CnwJLKpmbRVy\n90LgMYJ9PRDsi7jIzL4d7gBvbmbDzeyr7r6BYP/EFDNra2ZNE3YktyLYEtkeLv9Rh+lK9BQEUpEf\nA/9MsOJ4mGAHcqTcfQtwGTAZ2AZ8HXgX2B9BjVMJ+tvfAxYT/PKuqr41wDsEK+j/KTN6AvA7C466\n+jnBSrhOdbj7F8BEYA7wD2AMQVgWj3+fYCtkXdgd07FMvSsJPp+pBGEyEhgV7i9IhvsIgiYr3Adz\nEUEwFBIcBPBjStcxV4Z/PyIIsBvD4ckE+y22AQuB55JUm9SA6cY0cqyy4EStTcAYd38j1fWIpCtt\nEcgxxcxGht0HzQh+XR4k+BUuIhFREMixZgiwlqB74f8BF7l7RV1DIpIE6hoSEYk5bRGIiMRcg7u4\nWIcOHTwzMzPVZYiINChLliz53N3LPRy7wQVBZmYmeXl5qS5DRKRBMbMKz55X15CISMwpCEREYk5B\nICIScwoCEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJOQWBiEjMKQhERGJOQSAiEnMKAhGRmFMQiIjE\nnIJARCTmFAQiIjEXWRCY2WNmttXM3q9gvJnZg2a2xsxWmFm/qGqJo9xcyMyERo2Cv7m5qa4oelpm\nLXO6inyZ3T2SB3AW0A94v4Lx5wPPAQacASyqznz79+/vNfXkk+4nneRuFvx98skaz6JBefJJ94wM\ndyh9ZGSk93JrmbXM6SpZywzkeUXr64pGJOMBZFYSBA8DYxOGPwQ6VzXPmgZBHP/hnHTSkctb/Djp\npFRXFh0ts5Y5XSVrmSsLglTuI+gCbEgYLgjbjmJm480sz8zyCgsLa/Qmv/gF7NlzZNuePUF7ulq/\nvmbt6UDLXHV7OtAyV91eGw1iZ7G7T3P3HHfPOfHEcu+9XKE4/sPp3r1m7elAy1x1ezrQMlfdXhup\nDIKNQLeE4a5hW1LF8R/O3XdDRsaRbRkZQXu60jIHtMzpp16WuaI+o2Q8qHwfwXc4cmfxO9WZp/YR\nVE/cdpC7a5m1zOkrGctMJfsILBiffGY2ExgGdAC2AL8Gmobh8yczM+CPwEhgD3C1u+dVNd+cnBzP\ny6tysiPk5gb7BNavD7YE7r4bxo2r0SxERBo0M1vi7jnljosqCKJSmyAQEYm7yoKgQewsFhGR6CgI\nRERiTkEgIhJzCgIRkZhTEIiIxJyCQEQk5hQEIiIxpyAQEYk5BYGISMwpCEREYk5BICIScwoCEZGY\nUxCIiMScgkBEJOYUBCIiMacgEBGJOQWBiEjMKQhERGJOQZDGDh2CgwdTXYWIHOuapLoAqbl9++Cz\nz2Dz5uCR+DyxbcsWOO44OPNMOOccGDEC+veHJvrWRSSBbl5/jHCHnTurXrlv3gzbtx/9+kaNoGNH\n6NwZOnUK/nbuDLt2wYIFsGJFMF2rVnD22UEwnHMOnH568FoRSW+V3bxevw0jdvgwFBZWvHJPbN+7\n9+jXN2tWunI/5RQYNqx0JZ+40j/xxMp/6RcWwquvwvz58Mor8Pe/B+0dOsDw4aXB0LMnmEXxSYjI\nsUpbBLW0f3+wAq9q5b5lS9BXX1abNkf/ei+7cu/cGdq2jWbFvGFDsKVQHAwFBUF7ly6l3UjnnAPd\nuiX/vUWk/lW2RaAgSOAOX35ZddfM5s3wj38c/Xqz4Jd52RV72ZV7p06QkRHJItSKO6xZE4RC8ePz\nz4NxJ59curUwfHjQ/SQiDY+CANixAz79tPJf75s3w549R7/2uOOq/uXeuXOwkkyHHbGHD8PKlaWh\n8Oqrwf4LgG98o3Rr4ayzgi0WETn2KQiA3/4WfvGLI9tatap65d65M7RrF+9+86IiWLq0NBjefDPY\nn9GoUXAUUvEWw5Ahx9aWjoiUUhAA+fnBr9zEFX6LFhEUGAP798OiRaX7F95+OwiLpk1h0KDSYBg4\nMNiaEpHUUxBIpHbvDrYSircYliwJ9jtkZMDQoaXB0LcvNG6c6mpF4klBIPVq+3Z47bXSYFi5Mmhv\n0yY4/LU4GE47Ld5dbiL1SUEgKfXZZ0eew7B2bdDesWNpKJxzDnztawoGkagoCOSYsm7dkecwbN4c\ntHfvXnoOw/DhwTkNIpIcKQsCMxsJPAA0Bh5193vKjO8OzADahtNMcvd5lc1TQZBe3OGjj4JAmD8/\nCIjiczROOaV0a2HYsOAsaBGpnZQEgZk1Bj4Cvg0UAIuBse6enzDNNOBdd59qZlnAPHfPrGy+CoL0\ndvhwcF2k4v0Lr70WXC8JoE+f0mAYOhRat05trSINSaquNTQAWOPua8MiZgGjgfyEaRwo/u/cBtgU\nYT3SADRqFKzw+/SBW28NLqOdl1caDFOmwOTJwdFH3/xm6cltgwbB8cenunqRhinKLYIxwEh3vzYc\n/j4w0N1vSJimM/Ai0A5oAXzL3ZeUM6/xwHiA7t279//0008jqVmOffv2wVtvle5feOed4FpOzZrB\n4MGlRyM1bRqc5V38KDtc9lHZeF2dVWrr0KHgHJuKHgcP1mx8dnZwUEVtpKprqDpBcGtYwx/MbBAw\nHfiGux+uaL7qGpJEX34Jb7xRusWwbFmw3yGZGjWqfYhUNb4ur00cH7ewcq/7SrY2K+Kajk/2v8Wp\nU+G662r32lR1DW0EEq9d2TVsS3QNMBLA3d8ys+ZAB2BrhHVJGmnVCs4/P3gAbNsG69enfgWQOH7P\nntq/XuquoiCvbkBnZNRP8FdnvlEdSRdlECwGeppZD4IAuBy4osw064ERwH+a2alAc6AwwpokzbVv\nHzzSgXuw87yqoDl4MPm/PBuCxo2rXnnGcWupNiILAncvMrMbgBcIDg19zN1XmtldQJ67zwV+DDxi\nZhMJdhxf5Q3txAaRiJgFK7vGjYN9ICJRifSiyeE5AfPKtN2e8DwfODPKGkREpHLaaBIRiTkFgYhI\nzCkIRERiTkEgIhJzCgIRkZhTEIiIxJyCQEQk5hQEIiIxpyAQEYk5BYGISMwpCEREYk5BICIScwoC\nEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJOQWBiEjMKQhERGJOQSAiEnMKAhGRmFMQiIjEnIJARCTm\nFAQiIjGnIBARiTkFgYhIzCkIRERiTkEgIhJzCgIRkZhTEIiIxFykQWBmI83sQzNbY2aTKpjmUjPL\nN7OVZvbnKOsREZGjNalspJndWtl4d59cyWsbA1OAbwMFwGIzm+vu+QnT9AR+Bpzp7tvNrGNNihcR\nkbqrNAiAVnWY9wBgjbuvBTCzWcBoID9hmn8Bprj7dgB331qH9xMRkVqoNAjc/c46zLsLsCFhuAAY\nWGaafwIws/8FGgN3uPvzZWdkZuOB8QDdu3evQ0kiIlJWVV1DD1Y23t1vSsL79wSGAV2B183sdHf/\nosz7TAOmAeTk5Hgd31NERBJU1TW0pA7z3gh0SxjuGrYlKgAWuftB4BMz+4ggGBbX4X1FRKQGquoa\nmlGHeS8GeppZD4IAuBy4osw0fwXGAo+bWQeCrqK1dXhPERGpoaq2CAAwsxOB24AsoHlxu7ufU9Fr\n3L3IzG4AXiDo/3/M3Vea2V1AnrvPDceda2b5wCHgp+6+rdZLIyIiNVatIABygaeA7wDXAf8MFFb1\nInefB8wr03Z7wnMHbg0fIiKSAtU9oay9u08HDrr7a+7+Q6DCrQEREWk4qrtFcDD8u9nMvgNsAk6I\npiQREalP1Q2C35hZG+DHwENAa2BiZFWJiEi9qVYQuPvfw6c7gOHRlSMiIvWtWvsIzGyGmbVNGG5n\nZo9FV5aIiNSX6u4s7p14tm94baC+0ZQkIiL1qbpB0MjM2hUPmNkJVH//goiIHMOquzL/A/CWmf0l\nHP4ecHc0JYmISH2q7s7iJ8wsj9JzBy5OvK+AiIg0XDW5Q9kJwG53/yNQGF5DSEREGrjqHjX0a4Jr\nDf0sbGoKPBlVUSIiUn+qu0VwETAK2A3g7puo293LRETkGFHdIDgQXiDOAcysRXQliYhIfapuEMw2\ns4eBtmb2L8DLwKPRlSUiIvWlukcN3Wtm3wZ2AqcAt7v7S5FWJiIi9aLaJ4WFK/6XAMyskZmNc/fc\nyCoTEZF6UWnXkJm1NrOfmdkfzexcC9xAcDvJS+unRBERiVJVWwT/BWwH3gKuBX4OGHChuy+LuDYR\nEakHVQXB19z9dAAzexTYDHR3932RVyYiIvWiqqOGiu9MhrsfAgoUAiIi6aWqLYJsM9sZPjfg+HDY\nCO493zrS6kREJHKVBoG7N66vQkREJDVqctE5ERFJQwoCEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJ\nOQWBiEjMKQhERGJOQSAiEnORBoGZjTSzD81sjZlNqmS6S8zMzSwnynpERORokQWBmTUGpgDnAVnA\nWDPLKme6VsDNwKKoahERkYpFuUUwAFjj7mvd/QAwCxhdznT/Bvw7oKuaioikQJRB0AXYkDBcELaV\nMLN+QDd3/5/KZmRm480sz8zyCgsLk1+piEiMpWxnsZk1AiYDP65qWnef5u457p5z4oknRl+ciEiM\nRBkEG4FuCcNdw7ZirYBvAK+a2TrgDGCudhiLiNSvKINgMdDTzHqY2XHA5cDc4pHuvsPdO7h7prtn\nAm8Do9w9L8KaRESkjMiCwN2LgBuAF4BVwGx3X2lmd5nZqKjeV0REaqaqW1XWibvPA+aVabu9gmmH\nRVmLiIiUT2cWi4jEnIJARCTmFAQiIjGnIBARiTkFgYhIzCkIRERiTkEgIhJzCgIRkZhTEIiIxJyC\nQEQk5hQEIiIxpyAQEYk5BYGISMwpCEREYk5BICIScwoCEZGYUxCIiMScgkBEJOYUBCIiMacgEBGJ\nOQWBiEjMKQhERGJOQSAiEnMKAhGRmFMQiIjEnIJARCTmFAQiIjGnIBARiTkFgYhIzCkIRERiLtIg\nMLORZvahma0xs0nljL/VzPLNbIWZvWJmJ0VZj4iIHC2yIDCzxsAU4DwgCxhrZlllJnsXyHH33sDT\nwH9EVY+IiJQvyi2CAcAad1/r7geAWcDoxAncfYG77wkH3wa6RliPiIiUI8og6AJsSBguCNsqcg3w\nXHkjzGy8meWZWV5hYWESSxQRkWNiZ7GZXQnkAL8vb7y7T3P3HHfPOfHEE+u3OBGRNNckwnlvBLol\nDHcN245gZt8CfgGc7e77a/NGBw8epKCggH379tWqUDl2NW/enK5du9K0adNUlyKStqIMgsVATzPr\nQRAAlwNXJE5gZn2Bh4GR7r61tm9UUFBAq1atyMzMxMzqUrMcQ9ydbdu2UVBQQI8ePVJdjkjaiqxr\nyN2LgBuAF4BVwGx3X2lmd5nZqHCy3wMtgb+Y2TIzm1ub99q3bx/t27dXCKQZM6N9+/ba0hOJWJRb\nBLj7PGBembbbE55/K1nvpRBIT/peRaJ3TOwsFhGR1IllEOTmQmYmNGoU/M3Nrdv8tm3bRp8+fejT\npw+dOnWiS5cuJcMHDhyo1jyuvvpqPvzww0qnmTJlCrl1LTY0ZMgQTjnlFLKzsxkwYAArVqxIynyL\nrVmzhj59+gDw8ssvc+GFFyZ1/iKSPJF2DR2LcnNh/HjYE57G9umnwTDAuHG1m2f79u1ZtmwZAHfc\ncQctW7bkJz/5yRHTuDvuTqNG5Wfv448/XuX7XH/99bUrsAJPPfUUffr04ZFHHuG2227juefKPY1D\nRNJc7LYIfvGL0hAotmdP0J5sa9asISsri3HjxnHaaaexefNmxo8fT05ODqeddhp33XVXybRDhgxh\n2bJlFBUV0bZtWyZNmkR2djaDBg1i69bggKpf/vKX3H///SXTT5o0iQEDBnDKKaewcOFCAHbv3s0l\nl1xCVlYWY8aMIScnpySkKjJo0CA2biw9sve5555j0KBB9OvXj8suu4zdu3cDsGjRIgYNGkR2djYD\nBw5kz549fPzxxwwdOpS+ffvSv39/Fi1alNTPUESiF7sgWL++Zu119cEHHzBx4kTy8/Pp0qUL99xz\nD3l5eSxfvpyXXnqJ/Pz8o16zY8cOzj77bJYvX86gQYN47LHHyp23u/POO+/w+9//viRUHnroITp1\n6kR+fj6/+tWvePfdd6us8fnnny/putm6dSv33HMPr7zyCkuXLqV379488MAD7Nu3j8svv5wpU6aw\nfPlyXnzxRZo1a0bnzp156aWXePfdd8nNzeWmm26qw6clIqkQu66h7t2D7qDy2qPw9a9/nZycnJLh\nmTNnMn36dIqKiti0aRP5+flkZR15Lb7jjz+e8847D4D+/fvzxhtvlDvviy++uGSadevWAfDmm29y\n2223AZCdnc1pp51WYW2XXXYZ+/fvZ+/evSVbDQsXLiQ/P5/BgwcDcODAAYYMGcKqVavo3r07/fr1\nA6BNmzYA7N+/nxtuuIHly5fTpEkTPv744xp9PiKSerHbIrj7bsjIOLItIyNoj0KLFi1Knq9evZoH\nHniA+fPns2LFCkaOHFnuMfLHHXdcyfPGjRtTVFRU7rybNWtW5TSVeeqpp/jkk08YO3YsN998MxBs\nZYwcOZJly5axbNky8vPzmTZtWoXz+MMf/kC3bt147733eOedd9i/v1Ynh4tICsUuCMaNg2nT4KST\nwCz4O21a7XcU18TOnTtp1aoVrVu3ZvPmzbzwwgtJf48zzzyT2bNnA/Dee++V2/WUyMz47W9/y+uv\nv87q1asZPHgwr732GmvXrgWCfQ6rV68mKyuL9evXs3Tp0pJlOXToEDt27KBz586YGTNmzMDdk75M\nIhKt2AUBBCv9devg8OHgb32EAEC/fv3IysqiV69e/OAHP+DMM89M+nvceOONbNy4kaysLO68806y\nsrJKunEqkpGRwcSJE7n33nv5yle+wvTp07nsssvIzs5m8ODBfPTRRzRr1oyZM2cyYcIEsrOzOffc\nc0u6hR599FGys7P55JNPSrZSRKThsIb2Cy4nJ8fz8vKOaFu1ahWnnnpqiio6thQVFVFUVETz5s1Z\nvXo15557LqtXr6ZJk4a7O0jfr0jdmdkSd88pb1zDXTtIuXbt2sWIESMoKirC3Xn44YcbdAiISPS0\nhkgzbdu2ZcmSJakuQ0QakFjuIxARkVIKAhGRmFMQiIjEnIJARCTmFARJMHz48KNODrv//vuZMGFC\npa9r2bIlAJs2bWLMmDHlTjNs2DDKHi5b1v3338+ehCvpnX/++XzxxRfVKb1Sd9xxR8kltbOyspg5\nc2ad51lWZmYmn3/+OVD6eYhI/VIQJMHYsWOZNWvWEW2zZs1i7Nix1Xr9V7/6VZ5++ulav3/ZIJg3\nbx5t27at9fwSTZw4kWXLlvHss8/yox/9iIMHDyZlviJy7Ei7w0dvuQWquOpyjfXpA+HVn8s1ZswY\nfvnLX3LgwAGOO+441q1bx6ZNmxg6dCi7du1i9OjRbN++nYMHD/Kb3/yG0aNHH/H6devWccEFF/D+\n+++zd+9err76apYvX06vXr3Yu3dvyXQTJkxg8eLF7N27lzFjxnDnnXfy4IMPsmnTJoYPH06HDh1Y\nsGABmZmZ5OXl0aFDByZPnlxy9dJrr72WW265hXXr1nHeeecxZMgQFi5cSJcuXXj22Wc5/vjjK1zG\nnj17kpGRwfbt2+nYsSMff/wx119/PYWFhWRkZPDII4/Qq1cvtmzZwnXXXVdyiYqpU6cyePBgLrzw\nQjZs2MC+ffu4+eabGV98EwgRSbm0C4JUOOGEExgwYADPPfcco0ePZtasWVx66aWYGc2bN2fOnDm0\nbt2azz//nDPOOINRo0ZVeC/eqVOnkpGRwapVq1ixYkXJ1T4B7r77bk444QQOHTrEiBEjWLFiBTfd\ndBOTJ09mwYIFdOjQ4Yh5LVmyhMcff5xFixbh7gwcOJCzzz6bdu3asXr1ambOnMkjjzzCpZdeyjPP\nPMOVV15Z4TIuXbqUnj170rFjRwDGjx/Pn/70J3r27MmiRYv413/9V+bPn89NN93E2WefzZw5czh0\n6BC7du0C4LHHHuOEE05g7969fPOb3+SSSy6hffv2df3oRSQJ0i4IKvvlHqXi7qHiIJg+fToQXM3z\n5z//Oa+//jqNGjVi48aNbNmyhU6dOpU7n9dff73kmv69e/emd+/eJeNmz57NtGnTKCoqYvPmzeTn\n5x8xvqw333yTiy66qOQKqBdffDFvvPEGo0aNokePHiW3kky8jHVZ9913H48//jgfffQRf/vb34Dg\n7OWFCxfyve99r2S64quOzp8/nyeeeAIIropafJ2jBx98kDlz5gCwYcMGVq9erSAQOUakXRCkyujR\no5k4cSJLly5lz5499O/fH4Dc3FwKCwtZsmQJTZs2JTMzs9xLT1flk08+4d5772Xx4sW0a9eOq666\nqlbzKZZ4cbjGjRsf0QWVaOLEifzkJz9h7ty5XHPNNXz88cccPnyYtm3bVnnns2KvvvoqL7/8Mm+9\n9RYZGRkMGzasTrWLSHJpZ3GStGzZkuHDh/PDH/7wiJ3EO3bsoGPHjjRt2pQFCxbwaXl3xUlw1lln\n8ec//xmA999/v+Sm8jt37qRFixa0adOGLVu2HHF/4VatWvHll18eNa+hQ4fy17/+lT179rB7927m\nzJnD0KFDa7V8o0aNIicnhxkzZtC6dWt69OjBX/7yFyDY6lm+fDkAI0aMYOrUqQAll6nesWMH7dq1\nIyMjgw8++IC33367VjWISDQUBEk0duxYli9ffkQQjBs3jry8PE4//XSeeOIJevXqVek8JkyYwK5d\nuzj11FO5/fbbS7YssrOz6du3L7169eKKK6444hLW48ePZ+TIkQwfPvyIefXr14+rrrqKAQMGMHDg\nQK699lr69u1b6+W7/fbbmTx5MocPHyY3N5fp06eX3AXt2WefBeCBBx5gwYIFnH766fTv35/8/HxG\njhxJUVERp556KpMmTeKMM86odQ0ikny6DLUc8/T9itRdZZeh1haBiEjMKQhERGIubYKgoXVxSfXo\nexWJXloEQfPmzdm2bZtWGmnG3dm2bRvNmzdPdSkiaS0tziPo2rUrBQUFFBYWproUSbLmzZvTtWvX\nVJchktbSIgiaNm1Kjx49Up8v3CkAAAWPSURBVF2GiEiDFGnXkJmNNLMPzWyNmU0qZ3wzM3sqHL/I\nzDKjrEdERI4WWRCYWWNgCnAekAWMNbOsMpNdA2x395OB+4B/j6oeEREpX5RbBAOANe6+1t0PALOA\n0WWmGQ3MCJ8/DYywii7LKSIikYhyH0EXYEPCcAEwsKJp3L3IzHYA7YHPEycys/FA8QXsd5nZh7Ws\nqUPZeceAljketMzxUJdlPqmiEQ1iZ7G7TwOm1XU+ZpZX0SnW6UrLHA9a5niIapmj7BraCHRLGO4a\ntpU7jZk1AdoA2yKsSUREyogyCBYDPc2sh5kdB1wOzC0zzVzgn8PnY4D5rrPCRETqVWRdQ2Gf/w3A\nC0Bj4DF3X2lmdwF57j4XmA78l5mtAf5BEBZRqnP3UgOkZY4HLXM8RLLMDe4y1CIiklxpca0hERGp\nPQWBiEjMxSIIzOwxM9tqZu+nupb6YmbdzGyBmeWb2UozuznVNUXNzJqb2Ttmtjxc5jtTXVN9MLPG\nZvaumf091bXUBzNbZ2bvmdkyM8ur+hUNn5m1NbOnzewDM1tlZoOSOv847CMws7OAXcAT7v6NVNdT\nH8ysM9DZ3ZeaWStgCXChu+enuLTIhGelt3D3XWbWFHgTuNnd305xaZEys1uBHKC1u1+Q6nqiZmbr\ngBx3j83JZGY2A3jD3R8Nj8LMcPcvkjX/WGwRuPvrBEclxYa7b3b3peHzL4FVBGdypy0P7AoHm4aP\ntP6lY2Zdge8Aj6a6FomGmbUBziI4yhJ3P5DMEICYBEHchVd17QssSm0l0Qu7SZYBW4GX3D3dl/l+\n4P8Dh1NdSD1y4EUzWxJefibd9QAKgcfDLsBHzaxFMt9AQZDmzKwl8Axwi7vvTHU9UXP3Q+7eh+BM\n9gFmlrZdgWZ2AbDV3ZekupZ6NsTd+xFc2fj6sOs3nTUB+gFT3b0vsBs46rL+daEgSGNhP/kzQK67\n/3eq66lP4abzAmBkqmuJ0JnAqLDPfBZwjpk9mdqSoufuG8O/W4E5BFc6TmcFQEHC1u3TBMGQNAqC\nNBXuOJ0OrHL3yamupz6Y2Ylm1jZ8fjzwbeCD1FYVHXf/mbt3dfdMgrPy57v7lSkuK1Jm1iI8+IGw\ne+RcIK2PBnT3z4ANZnZK2DQCSOpBHw3i6qN1ZWYzgWFABzMrAH7t7tNTW1XkzgS+D7wX9pkD/Nzd\n56Wwpqh1BmaEN0VqBMx291gcUhkjXwHmhLctaQL82d2fT21J9eJGIDc8YmgtcHUyZx6Lw0dFRKRi\n6hoSEYk5BYGISMwpCEREYk5BICIScwoCEZGYUxCIhMzsUHhFy+JH0s7eNLPMOF39VhqWWJxHIFJN\ne8PLU4jEirYIRKoQXv/+P8Jr4L9jZieH7ZlmNt/MVpjZK2bWPWz/ipnNCe+LsNzMBoezamxmj4T3\nSngxPPsZM7spvG/ECjOblaLFlBhTEIiUOr5M19BlCeN2uPvpwB8JrvgJ8BAww917A7nAg2H7g8Br\n7p5NcE2YlWF7T2CKu58GfAFcErZPAvqG87kuqoUTqYjOLBYJmdkud29ZTvs64Bx3XxteyO8zd29v\nZp8T3PznYNi+2d07mFkh0NXd9yfMI5Pgstg9w+HbgKbu/hsze57gxkl/Bf6acE8FkXqhLQKR6vEK\nntfE/oTnhyjdR/cdYArB1sNiM9O+O6lXCgKR6rks4e9b4fOFBFf9BBgHvBE+fwWYACU3ymlT0UzN\nrBHQzd0XALcBbYCjtkpEoqRfHiKljk+4UivA8+5efAhpOzNbQfCrfmzYdiPBXaN+SnAHqeIrQt4M\nTDOzawh++U8ANlfwno2BJ8OwMODBZN+GUKQq2kcgUoU43ixd4kVdQyIiMactAhGRmNMWgYhIzCkI\nRERiTkEgIhJzCgIRkZhTEIiIxNz/AbiCfoeQMb5sAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZgV5Zn38e+PRQFFUMHRANoYFdkX\nW9QRd0XUuMWVwRjcUN64YCaJvBqjcfQ1kzhqXFBxmyQQkKgocYlGo1EmijQKqCjCkFZZVCBBBVRs\nuN8/qro93fRygD7d0PX7XNe5+tRTT1Xddbq77lPPU/WUIgIzM8uuZo0dgJmZNS4nAjOzjHMiMDPL\nOCcCM7OMcyIwM8s4JwIzs4xzIrBKJDWXtFLSrvVZtzFJ2kNSQa6TrrpuSc9KGlaIOCRdLenujV2+\nsUn6vqSn86h3n6QrGyImS8j3EWzZJK3MmWwDfAWsTacvjIjxDR/V5kXSHsC8iFA1854DXoqI66qU\nnwLcBuwaEWurLpfPujex7pHAfRFRVFfdTSXpfOAe4AtgHfC/wFUR8VSht22bB58RbOEiYtvyF/AB\ncHxO2XpJQFKLho9ys/Yb4HvVlH8PGFdbEmhiXk7/hrYHfgv8QVK7qpX899M0ORE0cZKul/SQpAmS\nPgfOknSApFclrZC0RNJtklqm9VtICklF6fS4dP7Tkj6X9IqkrhtaN51/jKT3JH0q6XZJ/yNpeA1x\n5xPjhZLmS/qnpNtylm0u6RZJyyUtAIbU8hE9Cuws6V9zlt8ROJbkgIikEyTNlPSZpA8kXV3L5z21\nfJ/qikPS+ZLeST+r/02/mZMegP8I7Jo2va2UtFP6u/zvnOVPlvR2+hn9RVK3nHkLJf1Q0pvp5z1B\n0ta1fA4ApInvAZKzy66SjpRUKulKSR8B9+Z8JrPSbU+V1Ctn27tJekzSUknLJP06Z39fTN83S3+n\nn6TxzZbUI503TtK1Oeu7KP09L0/Xu0taXuvfgeXPiSAbTgZ+D7QDHgLKgMuADsCBJAeoC2tZ/t+A\nq4EdSM46/mND60raCZgE/Djd7t+BgbWsJ58YjwX2AfqTJLgj0/KRwGCgL7AvcHpNG4mIVcDDwNk5\nxWcCsyPi7XR6JTAMaA8cD1wm6Tu1xF6urjg+Bo4DtgMuAG6X1CciPk2380HO2d0nuQtK6g78DrgE\n6Ag8B0wpT5ap04GjgN1JPqfqznwqUfKN/zzgc5ImIoDOwLbArsD/kbQvSUI4H9iRJHE8LmmrdPkn\ngflAEdCF5Pde1THA/sCeJGchZwL/qCaewcB1wKlAJ2AxUPVMt6a/A8uTE0E2TI2IP0bEuoj4IiKm\nR8S0iCiLiAXAWOCQWpZ/OCJKIuJrkn/CfhtR9zvAzIh4PJ13C7CsppXkGeONEfFpRJQCL+Zs63Tg\nlohYGBHLgV/UEi8kzUOn53xjPjstK4/lLxHxdvr5zQImVhNLdWqNI/2dLIjEX4DngYPyWC8kB84p\naWxfp+tuB+yXU+fWiPgo3fYT1P57GyRpBfARyUH3pIj4PJ1XBlwbEWsi4gtgBDAm/R2tjYgH0nr7\nAgeQJO8rImJV+vf2P9Vs72uSBLh3+lnMiYiPqqk3jKSvZGZEfAmMBg6R1DmnTk1/B5YnJ4Js+DB3\nQtLekp6U9JGkz0i+cXWoZfncf9DVJN8ON7Tut3LjiOQqhYU1rSTPGPPaFvB+LfEC/BX4DDhe0l4k\n3ywn5MRygKQX06aOT0m+Cdf2eZWrNQ5J35E0TdI/0oPw4DzXW77uivVFxDqSz7NTTp0N+b1NjYj2\nEdEhIv41TUzlPo6INTnTuwFXpM1CK9LYd0m33QUoratvJSKeBe4G7gI+lnS3pLZ57OdnwD83YT+t\nGk4E2VD10rB7gLeAPSJiO+BnQJ1XsmyiJSRNDABIEpX/mavalBiXkByQytV6eWualH5LcibwPeCp\niMg9W5kIPAJ0iYh2wH15xlJjHJJakzRJ3Qj8S0S0B57NWW9dl/MtJjkgl6+vGcnnuyiPuDZU1Vg+\nBH6eJo7yV5uImJTO201S8zpXGnFrRAwAegE9gB9WU63qfrYlaUoqxH5mlhNBNrUFPgVWpW3NtfUP\n1JcngAGSjk/bkS8jadsuRIyTgFGSOqUdv1fkscxvSfohziWnWSgnln9ExJeS9idpltnUOLYGtgKW\nAmvTPocjcuZ/DHSo4Vty+bpPkHRo2i/wY5J2/Wl5xrYp7gV+IGlfJbZNf6/bAK8Ay4H/J6mNpNaS\nDqy6AkkD01cLYBWwhuTS1aomAOdJ6pM23d1IcoVTjWeTtuGcCLLp34Hvkxw47iHpQC6oiPgYOAO4\nmeRA8W3gDZL7Huo7xrtI2tvfBKaTfPOuK775wGskB+gnq8weCdyo5KqrK6m+83OD4oiIFcDlwGSS\nTtJTSZJl+fy3SM5CStPml52qxPs2yedzF0kyGQKckPYXFFREvErymdxF0kzzHnBWOq+MpD+oO8nZ\nwQck+1ZVe+B+YAVQSnL2dHM12/oTSbPg5LTOriT9BlaPfEOZNYq06WAxcGpEvNzY8Zhlmc8IrMFI\nGiKpfXqKfzXJlSOvNXJYZpnnRGANaRCwgKQp42jg5IioqWnIzBqIm4bMzDLOZwRmZhm3xQ0g1aFD\nhygqKmrsMMzMtigzZsxYFhHVXrK9xSWCoqIiSkpKGjsMM7MtiqQa77B305CZWcY5EZiZZZwTgZlZ\nxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5\nEZiZZZwTgZlZxhUsEUh6QNInkt6qYb4k3SZpvqTZkgYUKpYsGj8eioqgWbPk5/jxjR1R4Xmfvc9N\nVcH3OSIK8gIOBgYAb9Uw/1jgaUDA/sC0fNa7zz77hNVu3LiINm0i4JtXmzZJeVPlffY+N1X1tc9A\nSdRwXC3ow+slFQFPRESvaubdA7wYERPS6bnAoRGxpLZ1FhcXh59QVruiIni/mmcR7bYblJY2dDQN\nw/v8De9z01Jf+yxpRkQUVzevMfsIOgEf5kwvTMvWI2mEpBJJJUuXLt3gDWXtVPKDDzasvCnwPtdd\n3hR4n+su3xhbRGdxRIyNiOKIKO7YsdpnL9do/HgYMSLJqBHJzxEjmnYy2HXXDStvCrzPdZc3Bd7n\nuss3RmMmgkVAl5zpzmlZvbrqKli9unLZ6tVJeVN1ww3Qpk3lsjZtkvKmyvuc8D43PQ2yzzV1HtTH\nCyii5s7i46jcWfxaPuvc0M5iqXInS/lL2qDVbHHGjYvYbbdkP3fbrWl3ppXzPnufm6r62Gcao7NY\n0gTgUKAD8DFwDdAyTT53SxJwBzAEWA2cExF19gJvaGdxFjuXzMyqqq2zuEWhNhoRQ+uYH8APCrX9\ncjfckPQJ5DYPNfVTSTOzDbFFdBZvimHDYOzY5AxASn6OHZuUm5lZAc8INifDhvnAb2ZWkyZ/RmBm\nZrVzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8u4TFw+apuHdevgq6+S15o137zf0LK66q5d+802\npfXfV1e2uc/fkGWyIHdAhOreb+7zN3ZdF14IQ4ZQ7zKTCKZOhWefTYairvqSqi+va96WsGzExh9Y\n6/Pg/NVXUFZWf79PCbbeev3XVltBi/SvenM8AGzM/A1ZJks2x4S9IfM3Zl2ff05BZCYRvPIKXH99\ndv9pNkXLltUfcHOnW7eGdu3qrldb+YbUbd48e9+CzQqloE8oK4RNfUJZ+fij69ZVflVXls+8zX1Z\n2LQDbsuWyZmFmW3ZGmXQuc2V9E3TiZmZ+aohM7PMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPL\nOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgn\nAjOzjHMiMDPLuIImAklDJM2VNF/S6Grm7yrpBUlvSJot6dhCxmNmZusrWCKQ1By4EzgG6AEMldSj\nSrWfApMioj9wJjCmUPGYmVn1CnlGMBCYHxELImINMBE4sUqdALZL37cDFhcwHjMzq0YhE0En4MOc\n6YVpWa5rgbMkLQSeAi6pbkWSRkgqkVSydOnSQsRqZpZZjd1ZPBT474joDBwL/E7SejFFxNiIKI6I\n4o4dOzZ4kGZmTVkhE8EioEvOdOe0LNd5wCSAiHgFaAV0KGBMZmZWRSETwXRgT0ldJW1F0hk8pUqd\nD4AjACR1J0kEbvsxM2tABUsEEVEGXAw8A7xDcnXQ25Kuk3RCWu3fgQskzQImAMMjIgoVk5mZra9F\nIVceEU+RdALnlv0s5/0c4MBCxmBmZrVr7M5iMzNrZE4EZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGZm\nGedEYGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGZmGedEYGaWcU4EZmYZ50RgZpZxTgRmZhnn\nRGBmlnFOBGZmGedEYGaWcU4EZmYZ50RgZpZxTgRmZhnnRGBmlnEt8q0oqROwW+4yEfFSIYIyM7OG\nk1cikPSfwBnAHGBtWhyAE4GZ2RYu3zOCk4BuEfFVIYMxM7OGl28fwQKgZSEDMTOzxpHvGcFqYKak\n54GKs4KIuLQgUZmZWYPJNxFMSV9mZtbE5JUIIuI3krYC9kqL5kbE14ULy8zMGkpefQSSDgXmAXcC\nY4D3JB2cx3JDJM2VNF/S6BrqnC5pjqS3Jf1+A2I3M7N6kG/T0H8BgyNiLoCkvYAJwD41LSCpOUni\nOApYCEyXNCUi5uTU2RP4v8CBEfFPSTtt3G6YmdnGyveqoZblSQAgIt6j7quIBgLzI2JBRKwBJgIn\nVqlzAXBnRPwzXe8necZjZmb1JN9EUCLpPkmHpq97gZI6lukEfJgzvTAty7UXsJek/5H0qqQh1a1I\n0ghJJZJKli5dmmfIZmaWj3wTwUiSu4ovTV9z0rJN1QLYEzgUGArcK6l91UoRMTYiiiOiuGPHjvWw\nWTMzK5fvVUNfATenr3wtArrkTHdOy3ItBKalVyD9XdJ7JIlh+gZsx8zMNkGtZwSSJqU/35Q0u+qr\njnVPB/aU1DW99PRM1r8X4TGSswEkdSBpKlqwEfthZmYbqa4zgsvSn9/Z0BVHRJmki4FngObAAxHx\ntqTrgJKImJLOGyypfDC7H0fE8g3dlpmZbTxFRN2VpG2ALyJiXXrp6N7A041xU1lxcXGUlNTVT21m\nZrkkzYiI4urm5dtZ/BLQKn0mwbPA94D/rp/wzMysMeWbCBQRq4HvAmMi4jSgZ+HCMjOzhpJ3IpB0\nADAMeDIta16YkMzMrCHlmwhGkQwFMTnt8N0deKFwYZmZWUPJ9z6CvwJ/zZleQHJjmZmZbeFqTQSS\nbo2IUZL+SPKM4koi4oSCRWZmZg2irjOC36U/byp0IGZm1jhqTQQRMSN9W0J6HwFUDDG9dYFjMzOz\nBpBvZ/HzQJuc6dbAc/UfjpmZNbR8E0GriFhZPpG+b1NLfTMz20LkmwhWSRpQPiFpH+CLwoRkZmYN\nKd9HVY4C/iBpMSBgZ+CMgkVlZmYNJt/7CKZL2hvolhbNbYwB58zMrP7l1TQkqQ1wBXBZRLwFFEna\n4KGpzcxs85NvH8GDwBrggHR6EXB9QSIyM7MGlW8i+HZE/BL4GiAdiVQFi8rMzBpMvolgjaTWpMNM\nSPo28FXBojIzswaT71VD1wB/ArpIGg8cCAwvVFBmZtZw6kwEkgS8S/JQmv1JmoQui4hlBY7NzMwa\nQJ2JICJC0lMR0ZtvHkpjZmZNRL59BK9L2regkZiZWaPIt49gP+AsSaXAKpLmoYiIPoUKzMzMGka+\nieDogkZhZmaNpq4nlLUCLgL2AN4E7o+IsoYIzMzMGkZdfQS/AYpJksAxwH8VPCIzM2tQdTUN9Uiv\nFkLS/cBrhQ/JzMwaUl1nBBUjjLpJyMysaarrjKCvpM/S9wJap9PlVw1tV9DozMys4Op6eH3zhgrE\nzMwaR743lJmZWRPlRGBmlnEFTQSShkiaK2m+pNG11DtFUkgqLmQ8Zma2voIlAknNgTtJ7j/oAQyV\n1KOaem2By4BphYrFzMxqVsgzgoHA/IhYEBFrgInAidXU+w/gP4EvCxiLmZnVoJCJoBPwYc70wrSs\ngqQBQJeIqHV4a0kjJJVIKlm6dGn9R2pmlmGN1lksqRlwM/DvddWNiLERURwRxR07dix8cGZmGVLI\nRLAI6JIz3TktK9cW6AW8mA5vvT8wxR3GZmYNq5CJYDqwp6SukrYCzgSmlM+MiE8jokNEFEVEEfAq\ncEJElBQwJjMzq6JgiSAdm+hi4BngHWBSRLwt6TpJJxRqu2ZmtmHyfTDNRomIp4CnqpT9rIa6hxYy\nFjMzq57vLDYzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcC\nM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOz\njHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4wr\naCKQNETSXEnzJY2uZv4PJc2RNFvS85J2K2Q8Zma2voIlAknNgTuBY4AewFBJPapUewMojog+wMPA\nLwsVj5mZVa+QZwQDgfkRsSAi1gATgRNzK0TECxGxOp18FehcwHjMzKwahUwEnYAPc6YXpmU1OQ94\nuroZkkZIKpFUsnTp0noM0czMNovOYklnAcXAr6qbHxFjI6I4Ioo7duzYsMGZmTVxLQq47kVAl5zp\nzmlZJZKOBK4CDomIrzZmQ19//TULFy7kyy+/3KhArWlq1aoVnTt3pmXLlo0ditlmrZCJYDqwp6Su\nJAngTODfcitI6g/cAwyJiE82dkMLFy6kbdu2FBUVIWlTYrYmIiJYvnw5CxcupGvXro0djtlmrWBN\nQxFRBlwMPAO8A0yKiLclXSfphLTar4BtgT9ImilpysZs68svv2THHXd0ErAKkthxxx19lmiWh0Ke\nERARTwFPVSn7Wc77I+trW04CVpX/Jszys1l0FpuZWePJZCIYPx6KiqBZs+Tn+PGbtr7ly5fTr18/\n+vXrx84770ynTp0qptesWZPXOs455xzmzp1ba50777yT8ZsabGrQoEF069aNvn37MmjQIObNm7fJ\n61y7di0HHXRQrXWOPvpoPv/8803elpnVH0VEY8ewQYqLi6OkpKRS2TvvvEP37t3zWn78eBgxAlav\n/qasTRsYOxaGDdv0+K699lq23XZbfvSjH1UqjwgigmbNNo/cO2jQIO644w769evHmDFjeO6553j0\n0Ucr1SkrK6NFi4K2HhbchvxtmDVlkmZERHF18zaPo1IDuuqqykkAkumrrqr/bc2fP58ePXowbNgw\nevbsyZIlSxgxYgTFxcX07NmT6667rqLuoEGDmDlzJmVlZbRv357Ro0fTt29fDjjgAD75JLmg6qc/\n/Sm33nprRf3Ro0czcOBAunXrxt/+9jcAVq1axSmnnEKPHj049dRTKS4uZubMmbXGefDBBzN//nwA\nOnfuzOjRo+nfvz+TJ09m3rx5HH300eyzzz4cfPDBvPfeewB89NFHnHjiifTp04e+ffsybdq0itgB\nFi1axKBBg+jXrx+9evWqiK9z586sWLECgF/+8pf06tWLXr16cfvtt1d8Zr169eK8886jZ8+eHHPM\nMe7wNSuwzCWCDz7YsPJN9e6773L55ZczZ84cOnXqxC9+8QtKSkqYNWsWf/7zn5kzZ856y3z66acc\ncsghzJo1iwMOOIAHHnig2nVHBK+99hq/+tWvKpLK7bffzs4778ycOXO4+uqreeONN+qM8Y9//CO9\ne/eumN5pp5144403OO200xgxYgRjxoxhxowZ3HjjjVx88cUA/OAHP+Coo45i9uzZzJgxY71v3ePG\njeP4449n5syZzJo1iz59+lSaP23aNMaPH8/06dN55ZVXGDNmDG+++SYAc+fOZdSoUbz99tu0bt2a\nxx57rM59MLONt2Wf92+EXXeF99+vvrwQvv3tb1Nc/M3Z2IQJE7j//vspKytj8eLFzJkzhx49Ko/F\n17p1a4455hgA9tlnH15++eVq1/3d7363ok5paSkAU6dO5YorrgCgb9++9OzZs8bYzjjjDFq3bs3u\nu+9e8Y28vBxgxYoVvPrqq5xyyikV88rKygB48cUXmThxIgAtWrRgu+22q5gHsO+++3LhhRfy5Zdf\nctJJJ9G3b99K2546dSqnnHIKrVu3BuCkk07i5ZdfZvDgweyxxx4ViSl338ysMDKXCG64ofo+ghtu\nKMz2ttlmm4r38+bN49e//jWvvfYa7du356yzzqq22WOrrbaqeN+8efNKB9hcW2+9dZ11avPQQw/R\nr1+/GmOOCDp06FBj01Jtl2cefvjhvPjiizz55JOcffbZ/OQnP2FYnp0w5fsFG79vZpa/zDUNDRuW\ndAzvthtIyc/66iiuy2effUbbtm3ZbrvtWLJkCc8880y9b+PAAw9k0qRJALz55pvVNj3la/vtt2eX\nXXZh8uTJAKxbt45Zs2YBcNhhh3H33XcDydVCn332WaVl33//fXbeeWdGjBjBOeecs14T1UEHHcTk\nyZP54osvWLlyJY8//nidVxyZWWFkLhFActAvLYV165KfDZEEAAYMGECPHj3Ye++9OfvssznwwAPr\nfRuXXHIJixYtokePHvz85z+nR48etGvXbqPXN3HiRO6+++6KZqYnnngCgDvuuINnnnmG3r17U1xc\nzLvvvltpueeff56+ffvSv39/Hn30US655JJK8wcOHMjQoUPZd9992X///Rk5cmSlfgozaziZu3y0\nqSsrK6OsrIxWrVoxb948Bg8ezLx587b4y0A3lv82zBK1XT6azaNDE7Zy5UqOOOIIysrKiAjuueee\nzCYBM8uPjxBNTPv27ZkxY0Zjh2FmW5BM9hGYmdk3nAjMzDLOicDMLOOcCMzMMs6JoB4cdthh690c\nduuttzJy5Mhal9t2220BWLx4Maeeemq1dQ499FCqXi5b1a233srqnFuljz322IqB3TbFtddeWzGk\ndq9evZgyZaMeILee888/v9Yb3X72s5/x3HPP1cu2zKxuTgT1YOjQoRXj7pSbOHEiQ4cOzWv5b33r\nWzz88MMbvf2qieCpp56qGAV0U11++eXMnDmTP/zhD5x77rmsW7eu0vyNGf7hvvvuW298pVzXXXcd\nRx5Zbw+vM7M6NLlEMGoUHHpo/b5Gjap9m6eeeipPPvlkxUNoSktLWbx4MQcddFDFdf0DBgygd+/e\nPP744+stX1paSq9evQD44osvOPPMM+nevTsnn3wyX3zxRUW9kSNHVgxhfc011wBw2223sXjxYg47\n7DAOO+wwAIqKili2bBkAN998c8VQz+VDWJeWltK9e3cuuOACevbsyeDBgyttpzrdu3enRYsWLFu2\njOHDh3PRRRex33778ZOf/IRVq1Zx7rnnMnDgQPr371+xj2vXruVHP/oRvXr1ok+fPhUD25Wf5axd\nu5bhw4fTq1cvevfuzS233ALA8OHDKxLj888/T//+/enduzfnnnsuX331VcU+XnPNNRWfa9U7m80s\nf76PoB7ssMMODBw4kKeffpoTTzyRiRMncvrppyOJVq1aMXnyZLbbbjuWLVvG/vvvzwknnFDjgG13\n3XUXbdq04Z133mH27NkMGDCgYt4NN9zADjvswNq1azniiCOYPXs2l156KTfffDMvvPACHTp0qLSu\nGTNm8OCDDzJt2jQigv32249DDjmE7bffnnnz5jFhwgTuvfdeTj/9dB555BHOOuusGvdx2rRpNGvW\njI4dOwKwcOFC/va3v9G8eXOuvPJKDj/8cB544AFWrFjBwIEDOfLII/ntb39LaWkpM2fOpEWLFvzj\nH/+otM6ZM2eyaNEi3nrrLYD1mrO+/PJLhg8fzvPPP89ee+3F2WefzV133cWoNDN36NCB119/nTFj\nxnDTTTdx33335fkbM7NcTS4RpF96G1x581B5Irj//vuBZATPK6+8kpdeeolmzZqxaNEiPv74Y3be\needq1/PSSy9x6aWXAtCnT59K4/hPmjSJsWPHUlZWxpIlS5gzZ8564/znmjp1KieffHLFaKLf/e53\nefnllznhhBPo2rVrxcijtQ31fMsttzBu3Djatm3LQw89VJHATjvtNJo3bw7As88+y5QpU7jpppuA\n5AD+wQcf8Nxzz3HRRRdV3Nm8ww47VFr37rvvzoIFC7jkkks47rjjGDx4cKX5c+fOpWvXruy1114A\nfP/73+fOO++sSAS5w3BXfbqameWvySWCxnLiiSdy+eWX8/rrr7N69Wr22WcfAMaPH8/SpUuZMWMG\nLVu2pKioaKOeuPX3v/+dm266ienTp7P99tszfPjwTXpyV9WhnmtqGrr88svXe+wmVB5eOyJ45JFH\n6Nat2wbFsP322zNr1iyeeeYZ7r77biZNmlTjQ3iqs6nDcJtZosn1ETSWbbfdlsMOO4xzzz23Uifx\np59+yk477UTLli154YUXeL+6p+LkOPjgg/n9738PwFtvvcXs2bOBZAjrbbbZhnbt2vHxxx/z9NNP\nVyzTtm3bah8If9BBB/HYY4+xevVqVq1axeTJkwsy1PPRRx/N7bffTvkAhuVDTh911FHcc889FQfp\nqk1Dy5YtY926dZxyyilcf/31vP7665Xmd+vWjdLS0orHaP7ud7/jkEMOqff4zbLOZwT1aOjQoZx8\n8smVriAaNmwYxx9/fMVwzXvvvXet6xg5ciTnnHMO3bt3p3v37hVnFuVDOu+999506dKl0hDWI0aM\nYMiQIXzrW9/ihRdeqCgfMGAAw4cPZ+DAgUBy2Wb//v3r/YlfV199NaNGjaJPnz6sW7eOrl278sQT\nT3D++efz3nvv0adPH1q2bMkFF1xQ8ahLSJ5rfM4551RciXTjjTdWWm+rVq148MEHOe200ygrK2Pf\nfffloosuqtfYzczDUFsT578Ns0Rtw1C7acjMLOOcCMzMMq7JJIItrYnLCs9/E2b5aRKJoFWrVixf\nvtz/+FYhIli+fDmtWrVq7E3yxO0AAAWySURBVFDMNntN4qqhzp07s3DhQpYuXdrYodhmpFWrVnTu\n3LmxwzDb7DWJRNCyZUu6du3a2GGYmW2RCto0JGmIpLmS5ksaXc38rSU9lM6fJqmokPGYmdn6CpYI\nJDUH7gSOAXoAQyVVHXv4POCfEbEHcAvwn4WKx8zMqlfIM4KBwPyIWBARa4CJwIlV6pwI/CZ9/zBw\nhGoaltPMzAqikH0EnYAPc6YXAvvVVCciyiR9CuwILMutJGkEMCKdXClp7kbG1KHqujPA+5wN3uds\n2JR93q2mGVtEZ3FEjAXGbup6JJXUdIt1U+V9zgbvczYUap8L2TS0COiSM905Lau2jqQWQDtgeQFj\nMjOzKgqZCKYDe0rqKmkr4Eyg6tPPpwDfT9+fCvwlfFeYmVmDKljTUNrmfzHwDNAceCAi3pZ0HVAS\nEVOA+4HfSZoP/IMkWRTSJjcvbYG8z9ngfc6GguzzFjcMtZmZ1a8mMdaQmZltPCcCM7OMy0QikPSA\npE8kvdXYsTQUSV0kvSBpjqS3JV3W2DEVmqRWkl6TNCvd5583dkwNQVJzSW9IeqKxY2kIkkolvSlp\npqSSupfY8klqL+lhSe9KekfSAfW6/iz0EUg6GFgJ/DYiejV2PA1B0i7ALhHxuqS2wAzgpIiY08ih\nFUx6V/o2EbFSUktgKnBZRLzayKEVlKQfAsXAdhHxncaOp9AklQLFEZGZm8kk/QZ4OSLuS6/CbBMR\nK+pr/Zk4I4iIl0iuSsqMiFgSEa+n7z8H3iG5k7vJisTKdLJl+mrS33QkdQaOA+5r7FisMCS1Aw4m\nucqSiFhTn0kAMpIIsi4d1bU/MK1xIym8tJlkJvAJ8OeIaOr7fCvwE2BdYwfSgAJ4VtKMdPiZpq4r\nsBR4MG0CvE/SNvW5ASeCJk7StsAjwKiI+Kyx4ym0iFgbEf1I7mQfKKnJNgVK+g7wSUTMaOxYGtig\niBhAMrLxD9Km36asBTAAuCsi+gOrgPWG9d8UTgRNWNpO/ggwPiIebex4GlJ66vwCMKSxYymgA4ET\n0jbzicDhksY1bkiFFxGL0p+fAJNJRjpuyhYCC3PObh8mSQz1xomgiUo7Tu8H3omImxs7noYgqaOk\n9un71sBRwLuNG1XhRMT/jYjOEVFEclf+XyLirEYOq6AkbZNe/EDaPDIYaNJXA0bER8CHkrqlRUcA\n9XrRxxYx+uimkjQBOBToIGkhcE1E3N+4URXcgcD3gDfTNnOAKyPiqUaMqdB2AX6TPhSpGTApIjJx\nSWWG/AswOX1sSQvg9xHxp8YNqUFcAoxPrxhaAJxTnyvPxOWjZmZWMzcNmZllnBOBmVnGORGYmWWc\nE4GZWcY5EZiZZZwTgVlK0tp0RMvyV73dvSmpKEuj39qWJRP3EZjl6Yt0eAqzTPEZgVkd0vHvf5mO\ngf+apD3S8iJJf5E0W9LzknZNy/9F0uT0uQizJP1ruqrmku5Nn5XwbHr3M5IuTZ8bMVvSxEbaTcsw\nJwKzb7Su0jR0Rs68TyOiN3AHyYifALcDv4mIPsB44La0/DbgrxHRl2RMmLfT8j2BOyOiJ7ACOCUt\nHw30T9dzUaF2zqwmvrPYLCVpZURsW015KXB4RCxIB/L7KCJ2lLSM5OE/X6flSyKig6SlQOeI+Cpn\nHUUkw2LvmU5fAbSMiOsl/YnkwUmPAY/lPFPBrEH4jMAsP1HD+w3xVc77tXzTR3cccCfJ2cN0Se67\nswblRGCWnzNyfr6Svv8byaifAMOAl9P3zwMjoeJBOe1qWqmkZkCXiHgBuAJoB6x3VmJWSP7mYfaN\n1jkjtQL8KSLKLyHdXtJskm/1Q9OyS0ieGvVjkidIlY8IeRkwVtJ5JN/8RwJLathmc2BcmiwE3Fbf\njyE0q4v7CMzqkMWHpVu2uGnIzCzjfEZgZpZxPiMwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLuP8P\nSZnCZYfcFrgAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SWaxiSJy2aqQ",
        "colab_type": "text"
      },
      "source": [
        "## Normalization\n",
        "\n",
        "**TODO**\n",
        "- reformat this text to fit the new focus on the Keras API and move low level api stuff into the extra notebook focusing on that!\n",
        "\n",
        "**Batch Normalization**\n",
        "\n",
        "Backpropagation means that the error gradients are propagated from the output layer to the input layer. Once the gradient of the cost function is computed it updates each parameter with a Gradient Descent step. However, gradients often get smaller as the algorithm progresses down to lower layers, therefore Gradient Descent leaves these lower layer connection weights mostly unchanged (*vanishing gradients*). This prevents training does not converge on a good solution. The opposite can also happen, in particular with recurrent neural networks, where gradients grow bigger so layers get large weight updates (*exploding gradients*)<sup>1</sup>.\n",
        "\n",
        "If we don't want the signal to die out or explode, then the variance of the output signal should be the same as the signal inputs and the gradients to have equal variance before and after flowing through a layer. The easiest way of doing this is to initialized the connection weights randomly using a strategy such as *Xavier initalization* (or *Glorot initialization*), or *He initalization*. By default *tf.layers.dense()* uses Xavier initalization with a uniform distribution, but can be changed using the *kernel_initializer* variable and the *tf.contrib.layers.variance_scaling_initializer()*. You can set the model to count the number of input connections, output, or the average (Xavier by default uses average)<sup>1</sup>.\n",
        "\n",
        "~~~\n",
        "he_init = tf.variance_scaling_initializer()\n",
        "hidden1 = tf.layers.dense(X, n_hidden1, activation=tf.nn.relu,\n",
        "                          kernel_initializer=he_init, name=\"hidden1\")\n",
        "~~~\n",
        "\n",
        "Batch Normalization also addresses the vanishing/exploding gradients and *Internal Covariate Shift* problem, where the distribution of each layers inputs changes during training as previous layers change. An operation is added before the activiation function of each layer to zero center and normalize the inputs, then scale and shift results to allow for the learning of the model to the optimal sclae and mean of the inputs to each layer. At test time the whole training set's mean and standard deviation is used.\n",
        "\n",
        "Batch Normalization is benificial as it means networks...\n",
        "- ...are less sensitive to weight initialization\n",
        "- ...are able to have larger learning weights\n",
        "- ...have improved accuracy\n",
        "- ...have a reduced need for a regulization techniques\n",
        "- ...does not need to have input data already normalized\n",
        "\n",
        "However it does add complexity so there is a runtime penalty at both training and testing phases<sup>2</sup>.\n",
        "\n",
        "The easiest way to impliment it is to use the tf.layers API\n",
        "\n",
        "Batch Norm and ELU are unlikely to have very positive impact in shallow networks so lets build a bigger one<sup>1</sup>\n",
        "\n",
        "**Gradient Clipping**\n",
        "\n",
        "A method to stop exploding gradients is to clip them during backpropogation so they do not exceed a threshold (useful for recurrent neural networks)<sup>1</sup>. In general Batch Normalization is now prefered.\n",
        "\n",
        "---\n",
        "1. Pascanu, R., Mikolov, T., & Bengio, Y. (2013, February). On the difficulty of training recurrent neural networks. In International conference on machine learning (pp. 1310-1318).\n",
        "2. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. https://github.com/ageron/handson-ml/blob/master/11_deep_learning.ipynb"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CLJFmGqvKAqq",
        "colab_type": "code",
        "outputId": "10d0e0c1-86d9-4d3a-8dca-6afde3eaf93e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 436
        }
      },
      "source": [
        "from tensorflow.keras.layers import Dense, Activation, BatchNormalization\n",
        "\n",
        "# Returns a short sequential model\n",
        "def create_batch_model(dense_layers=2, units_first=300, units_hidden=100):\n",
        "  model = Sequential()\n",
        "\n",
        "  for dense_layer in range(dense_layers):\n",
        "      if dense_layer == 0:\n",
        "        model.add(Dense(units=units_first,\n",
        "                        input_dim=X_train.shape[1]))\n",
        "      else:\n",
        "        model.add(Dense(units=units_hidden))\n",
        "      \n",
        "      model.add(BatchNormalization())\n",
        "      model.add(Activation('relu'))\n",
        "\n",
        "  model.add(Dense(units=1,\n",
        "                  activation='sigmoid'))\n",
        "\n",
        "  model.compile(optimizer=Adam(0.001),\n",
        "                loss=loss,\n",
        "                metrics=['accuracy', 'AUC', 'Recall', 'Precision'])\n",
        "\n",
        "  return model\n",
        "\n",
        "clear_session()\n",
        "# Create a basic model instance\n",
        "model = create_batch_model()\n",
        "model.summary()"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 300)               389700    \n",
            "_________________________________________________________________\n",
            "batch_normalization (BatchNo (None, 300)               1200      \n",
            "_________________________________________________________________\n",
            "activation (Activation)      (None, 300)               0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 100)               30100     \n",
            "_________________________________________________________________\n",
            "batch_normalization_1 (Batch (None, 100)               400       \n",
            "_________________________________________________________________\n",
            "activation_1 (Activation)    (None, 100)               0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1)                 101       \n",
            "=================================================================\n",
            "Total params: 421,501\n",
            "Trainable params: 420,701\n",
            "Non-trainable params: 800\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xNxSWVJ322LB",
        "colab_type": "text"
      },
      "source": [
        "## Regularization\n",
        "\n",
        "We have already seen some regularization methods, however here are some more...\n",
        "\n",
        "**l1 and l2 Regularization**\n",
        "\n",
        "Alike to other linear models you can use l1 and l2 Regularization to constrain the networks connection weights. This can be done by adding the terms to the cost function.\n",
        "\n",
        "**Max-Norm Regularization**\n",
        "\n",
        "For each neuron, the weights of the input connections are constrained, with clipping if needed.\n",
        "\n",
        "*Advantages*\n",
        "- help reduce overfitting\n",
        "- alleviate vanishing/exploding gradients (if not using batch normalization)\n",
        "\n",
        "**Dropout<sup>2,3</sup>**\n",
        "\n",
        "This is the most popular regularization method as it has been shown to improve the accuracy of start-of-the-art models. At each training step, every input neuron has a probability of beinf temporarily 'dropped out'. Dropout is typically set around 50%. This dropout only happens at training. Dropout means the network cannot rely on a few input neurons meaning it is more robust and can generalize better.\n",
        "\n",
        "One important thing to note is that if the proability of dropping is set at 50%, that means during testing there could be twice as many inputs. This means we need to multiply the weights by 0.5 after training to account for this. \n",
        "\n",
        "Monte-Carlo Dropout (MC Dropout) essentially means applying dropout during predictions (not just training). Generally makes predictions less certain.\n",
        "\n",
        "Lets do a model with dropout.\n",
        "\n",
        "---\n",
        "2. Hinton, G. E., Srivastava, N., & Krizhevsky, A. (2015). Improving neural networks by preventing co-adaption of feature detectors [R/OL].\n",
        "3. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929-1958."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mFqFnrWxKBuZ",
        "colab_type": "code",
        "outputId": "b4a3d447-094b-42e8-f1aa-ed41ca4b79b9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 360
        }
      },
      "source": [
        "from tensorflow.keras.layers import Dense, Activation, Dropout\n",
        "\n",
        "class MCDropout(Dropout):\n",
        "  def call(self, inputs):\n",
        "    return super().call(inputs, training=True)\n",
        "\n",
        "# Returns a sequential model\n",
        "def create_dropout_model(dense_layers=2, units_first=300, units_hidden=100, \n",
        "                         mcdropout = False, dropout_rate = 0.5):\n",
        "  model = Sequential()\n",
        "\n",
        "  for dense_layer in range(dense_layers):\n",
        "      if dense_layer == 0:\n",
        "        model.add(Dense(units=units_first,\n",
        "                        input_dim=X_train.shape[1],\n",
        "                        activation='relu'))\n",
        "      else:\n",
        "        model.add(Dense(units=units_hidden,\n",
        "                        activation='relu'))\n",
        "      \n",
        "      if mcdropout:\n",
        "        model.add(MCDropout(rate=dropout_rate))\n",
        "      else:\n",
        "        model.add(Dropout(rate=dropout_rate))\n",
        "\n",
        "  model.add(Dense(units=1,\n",
        "                  activation='sigmoid'))\n",
        "\n",
        "  model.compile(optimizer=Adam(0.001),\n",
        "                loss=loss,\n",
        "                metrics=['accuracy', 'AUC', 'Recall', 'Precision'])\n",
        "\n",
        "  return model\n",
        "\n",
        "clear_session()\n",
        "# Create a basic model instance\n",
        "model = create_dropout_model()\n",
        "model.summary()"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 300)               389700    \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 300)               0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 100)               30100     \n",
            "_________________________________________________________________\n",
            "dropout_1 (Dropout)          (None, 100)               0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1)                 101       \n",
            "=================================================================\n",
            "Total params: 419,901\n",
            "Trainable params: 419,901\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RlrG_Yy8Apsh",
        "colab_type": "text"
      },
      "source": [
        "# Model Design\n",
        "\n",
        "The number of possible hyperparameters means it can be hard to find the best way to set up a neural net. Options include a manual search, randomized search, or a tool such as Oscar (http://oscar.calldesk.ai/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f4Lienxt28Pd",
        "colab_type": "text"
      },
      "source": [
        "## Manual Search\n",
        "\n",
        "To start with, try a single hidden layer as this can give reasonable results if given enough neurons. However it is faster to train a deeper net with fewer neurons due to them having higher parameter efficency. You can gradually increase the number of hidden layers until you overfit the training set<sup>1</sup>.\n",
        "\n",
        "The input and output neurons is determained by the task (e.g. data length and classes to predict). Hidden layers are often formed into a funnel getting thinner towards the output. This is to account for low level features being formed into higher-level features. However it is also common that you may use the same number of neurons for each layer, meaning thats only 1 hyperparameter instead of one per layer. The number of neurons therefore becomes the what is changed. Another approach is to pcik a model with lots of layers and neurons and use early stopping or regulation techniques, such as dropout, to prevent overfitting<sup>1</sup>.\n",
        "\n",
        "For activation functions in the hidden layers, in most cases ReLU is a faster and less likely to get stuck on plateaus during Gradient Descent as other activation functions. A softmax activation function is generally good for mutually exclusive classification tasks. If they are not mutually exclusive or there are just two classes, the logistic function is better. For regression tasks, dont use one<sup>1</sup>.\n",
        "\n",
        "As proposed by Géron, A. (2017)<sup>1</sup>\n",
        "\n",
        "| stuff                  | stuff 2                      |\n",
        "|------------------------|------------------------------|\n",
        "| Initalization          | He Initialization            |\n",
        "| Activation Function    | ELU                          |\n",
        "| Normalization          | Batch Normalization          |\n",
        "| Regularization         | Dropout                      |\n",
        "| Optimizer              | Nestrov Accelerated Gradient |\n",
        "| Learning Rate Schedule | None                         |\n",
        "\n",
        "When building a model you have to decide on a number of layers, their type, and size. Burkov (2019)<sup>2</sup> recommends starting with one or two layers and increasing the size of each layer and the number of layers until the model fits the training data well. Once fitted well to the training data, but not the validation data, add regularisation to the model, increasing the network again if regularisation has affected training data fitting. This is continued until both training and validation models fit well.\n",
        "\n",
        "**NOTE**\n",
        "- This time I write it out in full rather than using functions just to show how little codes is actually required when using Keras for building a MLP model\n",
        "\n",
        "---\n",
        "1. Géron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. \" O'Reilly Media, Inc.\".\n",
        "2. Burkov, A. (2019). The Hundred - Page Machine Learning Book. Andriy Burkov."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_4xrFxri3Fuw",
        "colab_type": "code",
        "outputId": "1c51f57f-fb9d-4d85-ba9c-8365f88e7f4f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "from tensorflow.keras.layers import Dense, Activation, Dropout, BatchNormalization\n",
        "from tensorflow.keras import optimizers\n",
        "\n",
        "best_model_filepath = os.path.join(CLASS_BACK, \"Best_Deep_MLP_Model.ckpt\")\n",
        "\n",
        "clear_session()\n",
        "\n",
        "model = Sequential()\n",
        "\n",
        "# Input layer\n",
        "model.add(Dense(units=300,  \n",
        "                input_dim=X_train.shape[1],\n",
        "                kernel_initializer='he_normal'))\n",
        "model.add(BatchNormalization())\n",
        "model.add(Activation('elu'))\n",
        "model.add(Dropout(rate=0.5,seed=RANDOM_STATE))\n",
        "\n",
        "# Hidden layers\n",
        "for hidden_layer in range(2):\n",
        "    model.add(Dense(units=50,\n",
        "                    kernel_initializer='he_normal'))\n",
        "    model.add(BatchNormalization())\n",
        "    model.add(Activation('elu'))\n",
        "    model.add(Dropout(rate=0.5,seed=RANDOM_STATE))\n",
        "\n",
        "# Output layer\n",
        "model.add(Dense(units=1,\n",
        "                kernel_initializer='he_normal'))\n",
        "model.add(BatchNormalization())\n",
        "model.add(Activation('sigmoid'))\n",
        "\n",
        "model.compile(optimizer=Adam(0.001),\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy', 'AUC', 'Recall', 'Precision'])\n",
        "\n",
        "model.summary()\n",
        "\n",
        "# Save the model under certain conditions\n",
        "callback_checkpoint = ModelCheckpoint(filepath=best_model_filepath,\n",
        "                                      monitor='val_loss',\n",
        "                                      verbose=0,\n",
        "                                      save_best_only=True)\n",
        "\n",
        "callback_early_stopping = EarlyStopping(monitor='val_loss',\n",
        "                                        patience=5, \n",
        "                                        verbose=1)\n",
        "\n",
        "callback_tensorboard = TensorBoard(log_dir='./bal_MLP_logs/',\n",
        "                                   histogram_freq=0,\n",
        "                                   write_graph=False)\n",
        "\n",
        "callback_reduce_lr = ReduceLROnPlateau(monitor='val_loss',\n",
        "                                       factor=0.1,\n",
        "                                       min_lr=1e-4,\n",
        "                                       patience=0,\n",
        "                                       verbose=1)\n",
        "\n",
        "training_generator, steps_per_epoch = balanced_batch_generator(X_train, \n",
        "                                                               y_train, \n",
        "                                                               sampler=RandomOverSampler(), \n",
        "                                                               batch_size=50, \n",
        "                                                               random_state=42)\n",
        "\n",
        "history = model.fit_generator(generator=training_generator,\n",
        "                              epochs=20,\n",
        "                              steps_per_epoch=steps_per_epoch,\n",
        "                              validation_data = (X_val, y_val),\n",
        "                              callbacks=[callback_early_stopping,\n",
        "                                         callback_checkpoint,\n",
        "                                         callback_tensorboard,\n",
        "                                         callback_reduce_lr],\n",
        "                              verbose=1)\n",
        "\n",
        "plot_progress(history.history)"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter\n",
            "WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1\n",
            "WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2\n",
            "WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay\n",
            "WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate\n",
            "WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.\n",
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 300)               389700    \n",
            "_________________________________________________________________\n",
            "batch_normalization (BatchNo (None, 300)               1200      \n",
            "_________________________________________________________________\n",
            "activation (Activation)      (None, 300)               0         \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 300)               0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 50)                15050     \n",
            "_________________________________________________________________\n",
            "batch_normalization_1 (Batch (None, 50)                200       \n",
            "_________________________________________________________________\n",
            "activation_1 (Activation)    (None, 50)                0         \n",
            "_________________________________________________________________\n",
            "dropout_1 (Dropout)          (None, 50)                0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 50)                2550      \n",
            "_________________________________________________________________\n",
            "batch_normalization_2 (Batch (None, 50)                200       \n",
            "_________________________________________________________________\n",
            "activation_2 (Activation)    (None, 50)                0         \n",
            "_________________________________________________________________\n",
            "dropout_2 (Dropout)          (None, 50)                0         \n",
            "_________________________________________________________________\n",
            "dense_3 (Dense)              (None, 1)                 51        \n",
            "_________________________________________________________________\n",
            "batch_normalization_3 (Batch (None, 1)                 4         \n",
            "_________________________________________________________________\n",
            "activation_3 (Activation)    (None, 1)                 0         \n",
            "=================================================================\n",
            "Total params: 408,955\n",
            "Trainable params: 408,153\n",
            "Non-trainable params: 802\n",
            "_________________________________________________________________\n",
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n",
            "Train for 562 steps, validate on 2411 samples\n",
            "Epoch 1/20\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "  9/562 [..............................] - ETA: 1:55 - loss: 0.4719 - accuracy: 0.8444 - AUC: 0.9083 - Recall: 0.8326 - Precision: 0.8479"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "561/562 [============================>.] - ETA: 0s - loss: 0.2991 - accuracy: 0.9804 - AUC: 0.9954 - Recall: 0.9800 - Precision: 0.9807WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "If using Keras pass *_constraint arguments to layers.\n",
            "INFO:tensorflow:Assets written to: ./Best_Deep_MLP_Model.ckpt/assets\n",
            "562/562 [==============================] - 11s 20ms/step - loss: 0.2990 - accuracy: 0.9804 - AUC: 0.9954 - Recall: 0.9801 - Precision: 0.9807 - val_loss: 0.2026 - val_accuracy: 0.9830 - val_AUC: 0.9932 - val_Recall: 0.8607 - val_Precision: 0.9301\n",
            "Epoch 2/20\n",
            "557/562 [============================>.] - ETA: 0s - loss: 0.1998 - accuracy: 0.9927 - AUC: 0.9984 - Recall: 0.9971 - Precision: 0.9885\n",
            "Epoch 00002: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.\n",
            "562/562 [==============================] - 6s 11ms/step - loss: 0.1994 - accuracy: 0.9928 - AUC: 0.9985 - Recall: 0.9971 - Precision: 0.9886 - val_loss: 0.2268 - val_accuracy: 0.9780 - val_AUC: 0.9902 - val_Recall: 0.7960 - val_Precision: 0.9302\n",
            "Epoch 3/20\n",
            "560/562 [============================>.] - ETA: 0s - loss: 0.1608 - accuracy: 0.9959 - AUC: 0.9989 - Recall: 0.9993 - Precision: 0.9926\n",
            "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0001.\n",
            "562/562 [==============================] - 6s 11ms/step - loss: 0.1607 - accuracy: 0.9959 - AUC: 0.9989 - Recall: 0.9993 - Precision: 0.9926 - val_loss: 0.2281 - val_accuracy: 0.9805 - val_AUC: 0.9905 - val_Recall: 0.8209 - val_Precision: 0.9375\n",
            "Epoch 4/20\n",
            "562/562 [==============================] - 6s 11ms/step - loss: 0.1536 - accuracy: 0.9963 - AUC: 0.9991 - Recall: 0.9995 - Precision: 0.9932 - val_loss: 0.2350 - val_accuracy: 0.9768 - val_AUC: 0.9896 - val_Recall: 0.7761 - val_Precision: 0.9341\n",
            "Epoch 5/20\n",
            "562/562 [==============================] - 6s 11ms/step - loss: 0.1465 - accuracy: 0.9970 - AUC: 0.9992 - Recall: 0.9999 - Precision: 0.9942 - val_loss: 0.2408 - val_accuracy: 0.9755 - val_AUC: 0.9887 - val_Recall: 0.7612 - val_Precision: 0.9329\n",
            "Epoch 6/20\n",
            "562/562 [==============================] - 6s 11ms/step - loss: 0.1399 - accuracy: 0.9970 - AUC: 0.9993 - Recall: 0.9996 - Precision: 0.9945 - val_loss: 0.2526 - val_accuracy: 0.9714 - val_AUC: 0.9869 - val_Recall: 0.7114 - val_Precision: 0.9286\n",
            "Epoch 00006: early stopping\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZhV1Znv8e+PSWQQFEk0IIJDIoMI\neMQBETHExqhwMURRMHEKajTase2WjjFJY7ht1I5GQ4xksGMsJUajIQ4huUo7JSiFA4pIgwhYiLFA\nQbEcKHjvH3sXnCpOTVCnTg2/z/Ocp85ee6j3nIL97rXXXmspIjAzM6uqTaEDMDOzpskJwszMcnKC\nMDOznJwgzMwsJycIMzPLyQnCzMxycoKwRiGpraRNkvo05LaFJOkgSXl5TrzqsSX9RdLkfMQh6RpJ\nP9/Z/Ws47gWS/qehj2uNxwnCckpP0BWvrZI+ylrOeaKqSURsiYguEbG6IbdtqiT9P0nfy1H+FUlr\nJLWtz/Ei4sSIKGqAuMZIWlnl2NdGxEW7emxreZwgLKf0BN0lIroAq4FTs8p2OFFJatf4UTZpvwHO\nzlF+NnBXRGxp5HjM6s0JwnaKpB9K+p2keyR9AEyRdLSk+ZI2SFor6RZJ7dPt20kKSX3T5bvS9Y9K\n+kDS3yX1q++26fqTJP2vpI2SbpX0jKRzqom7LjFeKGm5pPck3ZK1b1tJN0laL2kFMLaGr+gPwD6S\njsnavwfwZeDOdHmcpBclvS9ptaRravi+n674TLXFkd7aWZJ+V69LuiAt7wb8CeiTVRv8TPq3/O+s\n/SdIWpx+R49L+kLWuhJJV0h6Of2+75G0Ww3fQ3Zcx0oqTvd7TtKRWevOl7QyjXmFpElp+eclPZnu\ns07S3XX5XdZAIsIvv2p8ASuBMVXKfgh8CpxKcqGxO3AEcCTQDjgA+F/g0nT7dkAAfdPlu4B1QAZo\nD/yO5Mq6vtt+BvgAGJ+uuwLYDJxTzWepS4x/BLoBfYF3Kz47cCmwGOgN9ACeTP4LVfu93QH8PGv5\nEqA4a/kEYGD6/R2WfsZT0nUHZR8beLriM9UWR/o3OQBQ+js+Agan68YAK3P8Lf87fd8f2JTu1x74\nDrAUaJ+uLwHmA/ukv/t/gQuq+fwXAP+Tvt8b2AicmX7PZwPrgT2BPdJ1B6fb7gsMSN//Hrgq/Y46\nAiMK/f+hNb1cg7Bd8XRE/CkitkbERxGxICKejYjyiFgBzAJG1bD/fRFRHBGbgSJgyE5sewrwYkT8\nMV13E8mJNqc6xvifEbExIlYC/5P1u04HboqIkohYD1xXQ7yQ3GY6PesK+2tpWUUsj0fE4vT7ewmY\nnSOWXGqMI/2brIjE48BjwMg6HBdgEjAnjW1zeuxuJEm1ws0R8Xb6ux+i5r9bhVOBxRFxT/rd/xZY\nAZxcETYwSFLHiFgbEa+m5ZtJEvW+EfFxRDxTx89hDcAJwnbFm9kLkg6R9LCktyW9D0wnuXKszttZ\n78uALjux7eey44iIILnKzamOMdbpdwGraogX4AngfeBUSZ8HhgL3ZMVytKT/kVQqaSPJFXdN31eF\nGuOQdIqkZyW9K2kDcGIdj1tx7G3Hi4itJN9nr6xt6vN3y3ncrLh7RcT7JDWLS4C3JT2Ufl8A/0JS\nkylOb2t9vY6fwxqAE4TtiqqPVt4OvAIcFBF7AN8juc2RT2tJbrUAIElUPplVtSsxrgX2y1qu8THc\nNFndSVJzOBt4JCKyazezgfuB/SKiG/DLOsZSbRySdgfuA/4T+GxEdAf+knXc2h6HfQvYP+t4bUi+\n3zV1iKvOx031qThuRDwaEWNIbi8tJ/k7kdYmLoiIfUkSyKzs9ifLLycIa0hdSe4lfyipP3BhI/zO\nh4Bhkk5V8iTV5UDPPMV4L/DPknqlDc5X1WGfO0kakc8j6/ZSVizvRsTHko4iub2zq3HsBnQASoEt\nkk4Bvpi1/h/A3pK61nDscZKOTxvv/5WkjefZOsZWnYeAgZLOSB8GOIukneVhSfumf79OJO1aHwJb\nASSdLqki4W8gSXB+AqyROEFYQ/oX4OskJ5TbSRqT8yoi/gGcAfyYpNHzQOAF4JM8xHgbyf38l4EF\nJFfqtcW3HHiO5MT9cJXVFwP/qeQpsO+QnJx3KY6I2AB8G3iApIF9IsnJuWL9KyS1lpXpU0qfqRLv\nYpLv5zaSJDMWGJe2R+y0iCgFxpEks/VpjKdExHtAW5JEtDZddwxJbQGSto8Fkj4keTLskmjG/WOa\nGyW1YLOWQUkHtLeAiRHxVKHjMWvOXIOwZk/SWEnd06eFriF58uW5Aodl1uw5QVhLcCzJI5OlwD8B\nEyKiultMZlZHvsVkZmY5uQZhZmY5tZgB1vbee+/o27dvocMwM2tWFi5cuC4icj4a3mISRN++fSku\nLi50GGZmzYqkakcE8C0mMzPLyQnCzMxycoIwM7OcWkwbhJk1rs2bN1NSUsLHH39c6FCsDjp27Ejv\n3r1p3759nffJa4KQNBb4CclYK7+MiOuqrL+IZMyVLSSTlEytGAde0r8D56frLouIufmM1czqp6Sk\nhK5du9K3b1+SQXStqYoI1q9fT0lJCf361X0w3LzdYkrHxJkJnAQMAM6UNKDKZndHxKERMQS4nmTA\nNdLtJpHMtjUW+JnqOcl7XRUVQd++0KZN8rNol6eFN2sdPv74Y3r06OHk0AxIokePHvWu7eWzDWI4\nsDyd2epTkrHvx2dvkE4UUqEz28eqHw/MjohPIuINkvHhhzd0gEVFMHUqrFoFEcnPqVOdJMzqysmh\n+diZv1U+E0QvKs96VXVWKgAkXSLpdZIaxGX13HdqOgl6cWlpab0DvPpqKCurXFZWlpSbmbV2BX+K\nKSJmRsSBJOPEf7ee+86KiExEZHr2rGmOmNxWVzOqfHXlZtZ0rF+/niFDhjBkyBD22WcfevXqtW35\n008/rdMxzj33XJYuXVrjNjNnzqSogW4rHHvssbz44osNcqzGkM9G6jVUnhaxtmkLZ5NMUrIz++6U\nPn2S20q5ys2sYRUVJbXz1auT/2MzZsDkyTt/vB49emw72f7gBz+gS5cuXHnllZW2iQgigjZtcl8L\n33HHHbX+nksuuaTWbVqqfNYgFgAHS+onqQNJo/Oc7A0kHZy1eDKwLH0/B5gkabd0/tmDycP4/jNm\nQKdOlcs6dUrKzazhNGZ73/LlyxkwYACTJ09m4MCBrF27lqlTp5LJZBg4cCDTp0/ftm3FFX15eTnd\nu3dn2rRpHHbYYRx99NG88847AHz3u9/l5ptv3rb9tGnTGD58OF/4whf429/+BsCHH37IV77yFQYM\nGMDEiRPJZDK11hTuuusuDj30UAYNGsR3vvMdAMrLyzn77LO3ld9yyy0A3HTTTQwYMIDBgwczZcqU\nBv/OqpO3GkRElEu6FJhL8pjrryNisaTpQHFEzAEulTSGZIKX90imOiTd7l7gVaCcZJrBBp+HtuLq\npSGvasxsRzW19+Xj/9trr73GnXfeSSaTAeC6665jr732ory8nNGjRzNx4kQGDKj8UOXGjRsZNWoU\n1113HVdccQW//vWvmTZt2g7Hjgiee+455syZw/Tp0/nzn//Mrbfeyj777MP999/PSy+9xLBhw2qM\nr6SkhO9+97sUFxfTrVs3xowZw0MPPUTPnj1Zt24dL7/8MgAbNmwA4Prrr2fVqlV06NBhW1ljyGsb\nREQ8EhGfj4gDI2JGWva9NDkQEZdHxMCIGBIRo9P5cCv2nZHu94WIeDRfMU6eDCtXwtatyU8nB7OG\n19jtfQceeOC25ABwzz33MGzYMIYNG8aSJUt49dVXd9hn991356STTgLg8MMPZ+XKlTmPfdppp+2w\nzdNPP82kSZMAOOywwxg4cGCN8T377LOccMIJ7L333rRv356zzjqLJ598koMOOoilS5dy2WWXMXfu\nXLp16wbAwIEDmTJlCkVFRfXq6LarCt5IbWYtX3Xtevlq7+vcufO298uWLeMnP/kJjz/+OIsWLWLs\n2LE5+wN06NBh2/u2bdtSXl6e89i77bZbrdvsrB49erBo0SJGjhzJzJkzufDCCwGYO3cuF110EQsW\nLGD48OFs2dLgN1RycoIws7wrZHvf+++/T9euXdljjz1Yu3Ytc+c2/KAMI0aM4N577wXg5ZdfzllD\nyXbkkUcyb9481q9fT3l5ObNnz2bUqFGUlpYSEXz1q19l+vTpPP/882zZsoWSkhJOOOEErr/+etat\nW0dZ1ft1eeKxmMws7wrZ3jds2DAGDBjAIYccwv7778+IESMa/Hd861vf4mtf+xoDBgzY9qq4PZRL\n7969ufbaazn++OOJCE499VROPvlknn/+ec4//3wiAkn86Ec/ory8nLPOOosPPviArVu3cuWVV9K1\na9cG/wy5tJg5qTOZTHjCILPGs2TJEvr371/oMJqE8vJyysvL6dixI8uWLePEE09k2bJltGvXtK7B\nc/3NJC2MiEyu7ZtW9GZmzdCmTZv44he/SHl5ORHB7bff3uSSw85o/p/AzKzAunfvzsKFCwsdRoNz\nI7WZmeXkBGFmZjk5QZiZWU5OEGZmlpMThJk1S6NHj96h09vNN9/MxRdfXON+Xbp0AeCtt95i4sSJ\nObc5/vjjqe2x+ZtvvrlSh7Uvf/nLDTJO0g9+8ANuvPHGXT5OQ3CCMLNm6cwzz2T27NmVymbPns2Z\nZ55Zp/0/97nPcd999+3076+aIB555BG6d+++08dripwgzKxZmjhxIg8//PC2yYFWrlzJW2+9xciR\nI7f1Sxg2bBiHHnoof/zjH3fYf+XKlQwaNAiAjz76iEmTJtG/f38mTJjARx99tG27iy++eNtQ4d//\n/vcBuOWWW3jrrbcYPXo0o0ePBqBv376sW7cOgB//+McMGjSIQYMGbRsqfOXKlfTv359vfOMbDBw4\nkBNPPLHS78nlxRdf5KijjmLw4MFMmDCB9957b9vvrxj+u2KQwCeeeGLbhElDhw7lgw8+2OnvtoL7\nQZjZLvvnf4aGnihtyBBIz6057bXXXgwfPpxHH32U8ePHM3v2bE4//XQk0bFjRx544AH22GMP1q1b\nx1FHHcW4ceOqnZf5tttuo1OnTixZsoRFixZVGq57xowZ7LXXXmzZsoUvfvGLLFq0iMsuu4wf//jH\nzJs3j7333rvSsRYuXMgdd9zBs88+S0Rw5JFHMmrUKPbcc0+WLVvGPffcwy9+8QtOP/107r///hrn\nd/ja177GrbfeyqhRo/je977Hf/zHf3DzzTdz3XXX8cYbb7Dbbrttu6114403MnPmTEaMGMGmTZvo\n2LFjPb7t3FyDMLNmK/s2U/btpYjgO9/5DoMHD2bMmDGsWbOGf/zjH9Ue58knn9x2oh48eDCDBw/e\ntu7ee+9l2LBhDB06lMWLF9c6EN/TTz/NhAkT6Ny5M126dOG0007jqaeeAqBfv34MGTIEqHlIcUjm\np9iwYQOjRo0C4Otf/zpPPvnkthgnT57MXXfdta3H9ogRI7jiiiu45ZZb2LBhQ4P05M5rDULSWOAn\nJBMG/TIirquy/grgApJJgUqB8yJiVbruepJZ5toAfwUuj5YycJRZC1PTlX4+jR8/nm9/+9s8//zz\nlJWVcfjhhwNQVFREaWkpCxcupH379vTt2zfnEN+1eeONN7jxxhtZsGABe+65J+ecc85OHadCxVDh\nkAwXXtstpuo8/PDDPPnkk/zpT39ixowZvPzyy0ybNo2TTz6ZRx55hBEjRjB37lwOOeSQnY4V8liD\nkNQWmAmcBAwAzpQ0oMpmLwCZiBgM3Adcn+57DDACGAwMAo4ARuUrVjNrnrp06cLo0aM577zzKjVO\nb9y4kc985jO0b9+eefPmsSrX5PNZjjvuOO6++24AXnnlFRYtWgQkQ4V37tyZbt268Y9//INHH90+\nd1nXrl1z3ucfOXIkDz74IGVlZXz44Yc88MADjBw5st6frVu3buy5557bah+//e1vGTVqFFu3buXN\nN99k9OjR/OhHP2Ljxo1s2rSJ119/nUMPPZSrrrqKI444gtdee63ev7OqfNYghgPLI2IFgKTZwHiS\naUQBiIh5WdvPBypuxgXQEegACGgPVF8/NLNW68wzz2TChAmVnmiaPHkyp556KoceeiiZTKbWK+mL\nL76Yc889l/79+9O/f/9tNZHDDjuMoUOHcsghh7DffvtVGip86tSpjB07ls997nPMm7f9VDZs2DDO\nOecchg8fDsAFF1zA0KFDa7ydVJ3f/OY3XHTRRZSVlXHAAQdwxx13sGXLFqZMmcLGjRuJCC677DK6\nd+/ONddcw7x582jTpg0DBw7cNjversjbcN+SJgJjI+KCdPls4MiIuLSa7X8KvB0RP0yXbyS5/STg\npxFxdU2/z8N9mzUuD/fd/NR3uO8m0UgtaQqQAW5Ilw8C+gO9gV7ACZJ2qKNJmiqpWFJxaWlpY4Zs\nZtbi5TNBrAH2y1runZZVImkMcDUwLiI+SYsnAPMjYlNEbAIeBY6uum9EzIqITERkevbs2eAfwMys\nNctnglgAHCypn6QOwCRgTvYGkoYCt5Mkh3eyVq0GRklqJ6k9SQP1kjzGamY7wQ8WNh8787fKW4KI\niHLgUmAuycn93ohYLGm6pHHpZjcAXYDfS3pRUkUCuQ94HXgZeAl4KSL+lK9Yzaz+OnbsyPr1650k\nmoGIYP369fXuPOc5qc1sp2zevJmSkpJd6hdgjadjx4707t2b9u3bVyr3nNRm1uDat29Pv379Ch2G\n5VGTeIrJzMyaHicIMzPLyQnCzMxycoIwM7OcnCDMzCwnJwgzM8vJCcLMzHJygjAzs5ycIMzMLCcn\nCDMzy8kJwszMcnKCMDOznJwgzMwsJycIMzPLyQnCzMxyymuCkDRW0lJJyyVNy7H+CkmvSlok6TFJ\n+2et6yPpL5KWpNv0zWesZmZWWd4ShKS2wEzgJGAAcKakAVU2ewHIRMRgkmlGr89adydwQ0T0B4YD\n72BmZo0mnzWI4cDyiFgREZ8Cs4Hx2RtExLyIKEsX5wO9AdJE0i4i/pputylrOzMzawT5TBC9gDez\nlkvSsuqcDzyavv88sEHSHyS9IOmGtEZSiaSpkoolFZeWljZY4GZm1kQaqSVNATLADWlRO2AkcCVw\nBHAAcE7V/SJiVkRkIiLTs2fPRorWzKx1yGeCWAPsl7XcOy2rRNIY4GpgXER8khaXAC+mt6fKgQeB\nYXmM1czMqshnglgAHCypn6QOwCRgTvYGkoYCt5Mkh3eq7NtdUkW14ATg1TzGamZmVeQtQaRX/pcC\nc4ElwL0RsVjSdEnj0s1uALoAv5f0oqQ56b5bSG4vPSbpZUDAL/IVq5mZ7UgRUegYGkQmk4ni4uJC\nh2Fm1qxIWhgRmVzrmkQjtZmZNT1OEGZmllO7QgdgZmY759134Zln4NNP4StfafjjO0GYmTUTb74J\nTz2VvJ5+Gl55JSk/9FAnCDOzViMCliypnBBWrUrWde0KxxwDkybBscfC8OH5icEJwsysCdi8GZ5/\nfntCeOYZWL8+WffZz8LIkfDtbyc/Bw+Gdo1w9naCMDMrgE2bYP787Qlh/nz46KNk3UEHwbhxSe1g\n5MhkWWr8GJ0gzMwawTvvJLWCioTwwguwZQu0aQOHHQbf+EaSEI49Fvbdt9DRJpwgzMwaWAS88UbS\nblCREJYuTdbtthsceSRMm5Ykg2OOgT32KGy81XGCMDPbRVu2JE8UZSeEt95K1nXvDiNGwLnnJreL\nDj88SRLNgROEmVk9ffIJLFiwPSE88wxs3Jis69ULjjsuSQYjR8LAgcltpObICcLMrBYbN8Lf/rY9\nITz3XJIkAA45BE4/fXtC2H//wjQo54MThJlZFWvXVu5/sGgRbN0KbdvCsGFwySVJMhgxAlryXGVO\nEGbWqkXAsmXbE8JTT8GKFcm6Tp3gqKPgmmuShHDkkdClS2HjbUxOEGbWqpSXw0svVa4hvJNOV9aj\nR/Jk0Te/mSSEoUOhffvCxltIThBm1qKVlSVtBhUJ4e9/TzqpAfTtCyeeuL394JBDWk77QUPIa4KQ\nNBb4CdAW+GVEXFdl/RXABUA5UAqcFxGrstbvQTLV6IMRcWk+YzWzluHdd5NaQUWD8sKFyTAWEgwa\nBGefvT0h9O5d6GibtrwlCEltgZnAl4ASYIGkORGRPbf0C0AmIsokXQxcD5yRtf5a4Ml8xWhmzUsE\nbNgAq1fv+Fq1Kvm5Zk2ybfv2cMQR28cvGjEC9tyzsPE3N/msQQwHlkfECgBJs4HxJDUCACJiXtb2\n84EpFQuSDgc+C/wZyDkdnpm1LJs3Jyf4qif97FfF7aEKHTpAnz7J60tfSsYtqhjhdPfdC/M5Wop8\nJohewJtZyyXAkTVsfz7wKICkNsB/kSSMMdXtIGkqMBWgT58+uxiumeVT9tV/rhP/6tVJ7+OIyvv1\n7Jmc/L/whSQBVCSD/fdPfvbs2Xw7ojV1TaKRWtIUklrCqLTom8AjEVGiGlqMImIWMAsgk8lEtRua\nWd59+mnlq/9cr9qu/rNP/H36JG0EnToV5vNYfhPEGmC/rOXeaVklksYAVwOjIiLtm8jRwEhJ3wS6\nAB0kbYqIaXmM18yqEQHvvVfzyb+2q/8TT9x+4q94+eq/actnglgAHCypH0limASclb2BpKHA7cDY\niHinojwiJmdtcw5JQ7aTg1me1HT1X3E76MMPK++z226w337JFX+uk/9++7kNoLnLW4KIiHJJlwJz\nSR5z/XVELJY0HSiOiDnADSQ1hN+nt5JWR8S4fMVk1hpVd/Wf3Q6wdu2OV/+f+Uxyou/fH/7pn3z1\n3xopqv6raKYymUwUFxcXOgyzgti8OTnhL19e+bViRfVX/1VP+L76b50kLYyInE+KNolGajOr3Sef\nJJPQVE0Cy5fDypXJnAQVOndOHvc85JAdr/733z+5+nePYauNE4RZE1JWllz150oCq1dXvg20xx5w\n8MGQycCkSUlCqHh99rNOALbrnCDMGtn778Prr1c++Vcsr6nynF+PHts7fmUngIMOStY5CVg+OUGY\n5cF77+WuBSxfvn3k0Ar77AMHHghjxlROAAce6KEhrLCcIMx2QgSsW1d9Enj33crb9+6dnPTHjauc\nBA44ALp2LcxnMKuNE4RZNSLg7berTwLvv7992zZtkgbggw5Kpp+smgT8RJA1R04Q1qpt3QolJTue\n/CvaBMrKtm/bti3065ec9I85pnIS6Ns3eXTUrCVxgrBmLyLpB/DJJ0mP4E8+yf3+gw92fEJoxYrt\nk89DMjbQgQcmrxNOqJwE+vRp3bOLWevjBGF1tnVr9Sffqu9rW9/Q7+tj99239xE45ZTKSaBXr6Sm\nYGZOEK3SE0/Ar36VDL1cnxNxdkeshtC+fXJbZrfdkiv36t536VLz+rq+79QpaQ/Yd18/HmpWF04Q\nrUQEPPYYTJ+eTMPYo0cynELFSbRzZ9hrr/qfdHf2ZN2hg0/SZk2dE0QLFwGPPgrXXgvz5ye3UG65\nBS64wE/WmFnNPBZjC7V1Kzz4YDIn78knJ6N1/vznydM53/qWk4OZ1c4JooXZuhV+/3sYOhQmTEja\nGX71K1i2DC680I9imlnd1SlBSLpc0h5K/ErS85JOzHdwVndbtsDdd8OgQUlHrU8+gd/+Fl57Dc47\nz49nmln91bUGcV5EvA+cCOwJnA1cV9tOksZKWippuaQdZoSTdIWkVyUtkvSYpP3T8iGS/i5pcbru\njHp8plZl82b47/9OJnWZPDl5RHP2bFi8GKZMgXZuZTKznVTXBFHxvMmXgd9GxOKsstw7SG2BmcBJ\nwADgTEkDqmz2Asl0ooOB+4Dr0/Iy4GsRMRAYC9wsqXsdY20VPv0UZs1K5vo999zkUdD774eXXoIz\nzvCz/Ga26+qaIBZK+gtJgpgrqSuwtZZ9hgPLI2JFRHwKzAbGZ28QEfMiomIwg/lA77T8fyNiWfr+\nLeAdoGcdY23RPv4YZs5MOnVdeGEy8cuf/gQLF8Jpp3kKSDNrOHW9AXE+MARYERFlkvYCzq1ln17A\nm1nLJcCRtfyOR6sWShoOdABez7FuKjAVoE+fPrWE07yVlSU1huuvT55IGjECfvlL+NKX3J/AzPKj\nrgniaODFiPhQ0hRgGPCThgoiPWYGGFWlfF/gt8DXI2KHGktEzAJmQTIndUPF05Rs2gQ/+xn8138l\n8wiMHg1FRXD88U4MZpZfdb0hcRtQJukw4F9IrubvrGWfNcB+Wcu907JKJI0BrgbGRcQnWeV7AA8D\nV0fE/DrG2WJs3Ag//GEyf/BVV8GQIUkP6McfT5KEk4OZ5VtdE0R5RARJG8JPI2ImUNs0JwuAgyX1\nk9QBmATMyd5A0lDgdpLk8E5WeQfgAeDOiLivjjG2CO++C9//fpIYrrkmGVZ6/nyYOzeZdtLMrLHU\n9RbTB5L+neTx1pGS2gA1PlkfEeWSLgXmAm2BX0fEYknTgeKImAPcAHQBfq/kknh1RIwDTgeOA3pI\nOic95DkR8WL9Pl7zUVoKN90EP/1pMiz1hAnw3e/CsGGFjszMWislFYNaNpL2Ac4CFkTEU5L6AMdH\nRG23mRpNJpOJ4uLiQodRb2+/DTfeCLfdBh99lHRyu/pqOPTQQkdmZq2BpIURkcm1rk63mCLibaAI\n6CbpFODjppQcmqM1a+Dyy5MZym66KXlEdfHipJObk4OZNQV1HWrjdOA54Kskt3+elTQxn4G1VKtW\nwcUXJ/MS/OxncNZZsHRpMixG//6Fjs7MbLu6tkFcDRxR0ZAsqSfw/0h6P1sdvP46/N//C3femTyB\ndN55MG1aMpexmVlTVNcE0Sb7KSNgPR4Jtk6WLoUZM5KB9Nq1g4sugn/7t2SyHjOzpqyuCeLPkuYC\n96TLZwCP5CekluGVV5LE8LvfJXMvXH45XHllMt2lmVlzUKcEERH/KukrwIi0aFZEPJC/sJqvF15I\nOrj94Q/JAHpXXQVXXJGMmWRm1pzUeTDoiLgfuD+PsTRrzz2XTOv50EPQrVvSye3yy5O5n83MmqMa\nE4SkD4BcHSUERETskZeompFnnkkSw9y5sOeeMH16MqVndw9ObmbNXI0JIiJqG06jVYqAJ55IEsPj\nj8Pee8N118E3vwld/Y2ZWTw/mZQAAA1TSURBVAvh+cbqIQL++tckMTz9NOyzTzLK6oUXQufOhY7O\nzKxhOUHUQQQ88kiSGJ59Fnr3hltvhfPPT55QMjNridyXoQZbt8IDD0AmA6eckoyb9POfw/LlcOml\nTg5m1rI5QeSwZQvce28yB8NppyVzM/z617BsWXI7abfdCh2hmVn+OUFkKS9PZmsbNAjOOAM2b07G\nSHrtNTj3XGhf4wDnZmYtixMESSK4445ksLwpU5IhMX73u6Q3dMWymVlr0+pPfa+/DmPGwMqVMHRo\n0gN6/Hho49RpZq1cXk+DksZKWippuaRpOdZfIelVSYskPSZp/6x1X5e0LH19PV8x9u0LRx2V9IBe\nuDCZyc3JwcwsjzUISW2BmcCXgBJggaQ5EfFq1mYvAJmIKJN0MXA9cIakvYDvAxmSntwL033fa+g4\n27aFe+6pfTszs9Ymn9fKw4HlEbEiIj4FZgPjszeIiHkRUZYuzgd6p+//CfhrRLybJoW/AmPzGGur\nUlSU1JzatEl+FhUVOiIza4rymSB6AW9mLZekZdU5H3i0PvtKmiqpWFJxaWnpLobbOhQVwdSpycx2\nEcnPqVOdJMxsR03ibrukKSS3k26oz34RMSsiMhGR6enxtOvk6quhrKxyWVlZUm5mli2fCWINkD1v\nWu+0rBJJY0imNB0XEZ/UZ1+rv9Wr61duZq1XPhPEAuBgSf0kdQAmAXOyN5A0FLidJDlkT2k6FzhR\n0p6S9gROTMtsF/XpU79yM2u98pYgIqIcuJTkxL4EuDciFkuaLmlcutkNQBfg95JelDQn3fdd4FqS\nJLMAmJ6W2S6aMQM6dapc1qlTUm5mlk0RueYDan4ymUwUFxcXOoxmoagoaXNYvTqpOcyYAZMnFzoq\nMysESQsjIpNrXavvSd0aTZ7shGBmtWsSTzGZmVnT4wRhZmY5OUGYmVlOThBmZpaTE4SZmeXkBGFm\nZjk5QZiZWU5OEGZmlpMThJmZ5eQEYWZmOTlBmJlZTk4QZmaWkxOEmZnl5ARhZmY55TVBSBoraamk\n5ZKm5Vh/nKTnJZVLmlhl3fWSFktaIukWScpnrGZmVlneEoSktsBM4CRgAHCmpAFVNlsNnAPcXWXf\nY4ARwGBgEHAEMCpfsZqZ2Y7yOWHQcGB5RKwAkDQbGA+8WrFBRKxM122tsm8AHYEOgID2wD/yGKuZ\nmVWRz1tMvYA3s5ZL0rJaRcTfgXnA2vQ1NyKWNHiEZmZWrSbZSC3pIKA/0JskqZwgaWSO7aZKKpZU\nXFpa2thhmpm1aPlMEGuA/bKWe6dldTEBmB8RmyJiE/AocHTVjSJiVkRkIiLTs2fPXQ7YzMy2y2eC\nWAAcLKmfpA7AJGBOHfddDYyS1E5Se5IGat9iMjNrRHlLEBFRDlwKzCU5ud8bEYslTZc0DkDSEZJK\ngK8Ct0tanO5+H/A68DLwEvBSRPwpX7GamdmOFBGFjqFBZDKZKC4uLnQYZmbNiqSFEZHJta5JNlKb\nmVnhOUGYmVlOThBmZpaTE4SZmeXkBGFmZjk5QZiZWU5OEGZmlpMThJmZ5eQEYWZmOTlBmJlZTk4Q\nZmaWkxOEmZnl5ARhZmY5OUGYmVlOThBmZpaTE4SZmeWU1wQhaaykpZKWS5qWY/1xkp6XVC5pYpV1\nfST9RdISSa9K6pvPWM3MrLK8JQhJbYGZwEnAAOBMSQOqbLYaOAe4O8ch7gRuiIj+wHDgnXzFai1b\nURH07Qtt2iQ/i4oKHZFZ89Auj8ceDiyPiBUAkmYD44FXKzaIiJXpuq3ZO6aJpF1E/DXdblMe47QW\nrKgIpk6FsrJkedWqZBlg8uTCxWXWHOTzFlMv4M2s5ZK0rC4+D2yQ9AdJL0i6Ia2RVCJpqqRiScWl\npaUNELK1NFdfvT05VCgrS8rNrGZNtZG6HTASuBI4AjiA5FZUJRExKyIyEZHp2bNn40ZozcLq1fUr\nN7Pt8pkg1gD7ZS33TsvqogR4MSJWREQ58CAwrIHjs1agT5/6lZvZdvlMEAuAgyX1k9QBmATMqce+\n3SVVVAtOIKvtwqyuZsyATp0ql3XqlJSbWc3yliDSK/9LgbnAEuDeiFgsabqkcQCSjpBUAnwVuF3S\n4nTfLSS3lx6T9DIg4Bf5itVarsmTYdYs2H9/kJKfs2a5gdqsLhQRhY6hQWQymSguLi50GGZmzYqk\nhRGRybWuqTZSm5lZgTlBmLVA7hxoDSGfHeXMrADcOdAaimsQZi2MOwdaQ3GCMGth3DnQGooThFkL\n486B1lCcIMxaGHcOtIbiBGHWwrhzoDUUP8Vk1gJNnuyEYLvONQgzM8vJCcLMWgR3Dmx4vsVkZs2e\nOwfmh2sQZtbsuXNgfjhBmFmz586B+eEEYWbNnjsH5ocThJk1e+4cmB95TRCSxkpaKmm5pGk51h8n\n6XlJ5ZIm5li/h6QSST/NZ5xm1ry5c2B+5O0pJkltgZnAl4ASYIGkORGRPbf0auAckulFc7kWeDJf\nMZpZy+HOgQ0vnzWI4cDyiFgREZ8Cs4Hx2RtExMqIWARsrbqzpMOBzwJ/yWOMZmbNVr77fuQzQfQC\n3sxaLknLaiWpDfBfVF+zqNhuqqRiScWlpaU7HaiZWXNT0fdj1SqI2N73oyGTRFNtpP4m8EhElNS0\nUUTMiohMRGR69uzZSKGZmRVeY/T9yGdP6jXAflnLvdOyujgaGCnpm0AXoIOkTRGxQ0O3mVlr1Bh9\nP/KZIBYAB0vqR5IYJgFn1WXHiNjW1CTpHCDj5GBmtl2fPsltpVzlDSVvt5giohy4FJgLLAHujYjF\nkqZLGgcg6QhJJcBXgdslLc5XPGZmLUlj9P1QRDTc0Qook8lEcXFxocMwM2s0RUVJm8Pq1UnNYcaM\n+j/qK2lhRGRyrfNormZmzVS++3401aeYzMyswJwgzMwsJycIMzPLyQnCzMxycoIwM7OcWsxjrpJK\ngRzdRupsb2BdA4XTXLS2z9zaPi/4M7cWu/KZ94+InGMVtZgEsaskFVf3LHBL1do+c2v7vODP3Frk\n6zP7FpOZmeXkBGFmZjk5QWw3q9ABFEBr+8yt7fOCP3NrkZfP7DYIMzPLyTUIMzPLyQnCzMxyavUJ\nQtKvJb0j6ZVCx9IYJO0naZ6kVyUtlnR5oWPKN0kdJT0n6aX0M/9HoWNqLJLaSnpB0kOFjqUxSFop\n6WVJL0pqFeP/S+ou6T5Jr0laIunoBjt2a2+DkHQcsAm4MyIGFTqefJO0L7BvRDwvqSuwEPg/EfFq\ngUPLG0kCOkfEJkntgaeByyNifoFDyztJVwAZYI+IOKXQ8eSbpJUkM1C2mo5ykn4DPBURv5TUAegU\nERsa4titvgYREU8C7xY6jsYSEWsj4vn0/Qcks/31KmxU+RWJTeli+/TV4q+MJPUGTgZ+WehYLD8k\ndQOOA34FEBGfNlRyACeIVk1SX2Ao8GxhI8m/9FbLi8A7wF8josV/ZuBm4N+ArYUOpBEF8BdJCyVN\nLXQwjaAfUArckd5K/KWkzg11cCeIVkpSF+B+4J8j4v1Cx5NvEbElIoYAvYHhklr07URJpwDvRMTC\nQsfSyI6NiGHAScAl6S3klqwdMAy4LSKGAh8C0xrq4E4QrVB6H/5+oCgi/lDoeBpTWv2eB4wtdCx5\nNgIYl96Tnw2cIOmuwoaUfxGxJv35DvAAMLywEeVdCVCSVSO+jyRhNAgniFYmbbD9FbAkIn5c6Hga\ng6Sekrqn73cHvgS8Vtio8isi/j0iekdEX2AS8HhETClwWHklqXP64AXpbZYTgRb9dGJEvA28KekL\nadEXgQZ74KRdQx2ouZJ0D3A8sLekEuD7EfGrwkaVVyOAs4GX03vyAN+JiEcKGFO+7Qv8RlJbkoui\neyOiVTz22cp8FngguQaiHXB3RPy5sCE1im8BRekTTCuAcxvqwK3+MVczM8vNt5jMzCwnJwgzM8vJ\nCcLMzHJygjAzs5ycIMzMLCcnCLNaSNqSjg5a8WqwnqqS+raWkYSt+Wn1/SDM6uCjdJgOs1bFNQiz\nnZTOPXB9Ov/Ac5IOSsv7Snpc0iJJj0nqk5Z/VtID6bwUL0k6Jj1UW0m/SOeq+Eva2xtJl6XzdiyS\nNLtAH9NaMScIs9rtXuUW0xlZ6zZGxKHAT0lGTwW4FfhNRAwGioBb0vJbgCci4jCS8XIWp+UHAzMj\nYiCwAfhKWj4NGJoe56J8fTiz6rgntVktJG2KiC45ylcCJ0TEinQAxLcjooekdSSTMm1Oy9dGxN6S\nSoHeEfFJ1jH6kgw/fnC6fBXQPiJ+KOnPJJNZPQg8mDWnhVmjcA3CbNdENe/r45Os91vY3jZ4MjCT\npLaxQJLbDK1ROUGY7Zozsn7+PX3/N5IRVAEmA0+l7x8DLoZtExh1q+6gktoA+0XEPOAqoBuwQy3G\nLJ98RWJWu92zRr4F+HNEVDzquqekRSS1gDPTsm+RzPD1rySzfVWMrnk5MEvS+SQ1hYuBtdX8zrbA\nXWkSEXBLQ04laVYXboMw20lpG0QmItYVOhazfPAtJjMzy8k1CDMzy8k1CDMzy8kJwszMcnKCMDOz\nnJwgzMwsJycIMzPL6f8D+hGl8N1RggAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd5wV5dn/8c+XooA0BYwJSEkkKm1l\nWUEjFmzBhrEiwRixkJhYk5j4S4wajXmSGI2a8PhIjC0iyKNiiyWCJMTHRClSFFSIojQVEBHFtnr9\n/pjZ9eyy5QB7dt2d7/v1mteeueeemWvOOTvXmXtm7lFEYGZm2dWsoQMwM7OG5URgZpZxTgRmZhnn\nRGBmlnFOBGZmGedEYGaWcU4EVoGk5pLeldS9Lus2JEm7SCrIddKVly3pb5JGFyIOST+X9D9bOr9Z\ndZwIGrl0R1w2fCrp/ZzxKndINYmITyKibUS8Vpd1P68kTZV0SRXlx0laIan55iwvIg6NiAl1ENfB\nkpZWWvYVEfHdrV22WWVOBI1cuiNuGxFtgdeAo3LKNtkhSWpR/1F+rt0GfKuK8m8Bd0TEJ/UcT+b4\nO9nwnAiaOEm/lHSXpImSNgAnS9pb0r8lvS1plaTrJbVM67eQFJJ6puN3pNMfkbRB0r8k9drcuun0\nwyS9JGm9pD9I+j9Jp1YTdz4xfkfSEknrJF2fM29zSb+XtFbSy8DwGt6ie4GdJH0tZ/5OwOHA7en4\nCElzJb0j6TVJP6/h/X6ybJtqi0PSGZIWpe/VfySdkZZ3AB4Euucc3e2Yfpa35sx/jKTn0/foCUm7\n5kxbLukHkhak7/dESdtWE3NvSdMlvSVpjaS/pDGUTe8h6T5Jq9Pp1+VM+46kF9JteE5SUeXvRVrv\nDkmXpa8PlrRU0k8lvQ78SVInSQ+n61gn6UFJXXM/E0m3pt+FdZLuSctfkHRYTr1t0+n9q/uMbFNO\nBNlwDHAn0AG4CygFzgM6A/uQ7KC+U8P83wR+DuxActRxxebWlbQjMBm4MF3vK8DgGpaTT4yHA4OA\ngSQJ7uC0/CzgUKAI2BM4sbqVRMR7wN3AKTnFJwHzI+L5dPxdYDTQETgKOE/SkTXEXqa2ON4AjgDa\nA2cCf5A0ICLWp+t5Lefo7s3cGSXtDvwFOAfoAkwFHihLlqkTgUOAL5O8T1Ud+QAI+CWwE9Anrf/z\ndD0tgL8CS4CewM4knyOSRgEXp+9Ne+BY4K083heAbkBboDvwPZJ90Z/S8R7Ax8B1OfXvBLZJ49sx\nZ9rtwMk59Y4ElkbEgjzjMICI8NBEBmApcHClsl8CT9Qy34+A/01ftwAC6JmO3wH8T07dEcBzW1D3\nNOCfOdMErAJOzXPbqopxr5zp9wI/Sl/PAM7ImXZ48lWvdtkHkOzAtk3HnwbOqaH+H4Gr0te75C4b\neLJsm7YgjoeA76evDybZoVX+LG9NX/8CuDNnWjPgdWBoOr4cOCln+jXAH/N8r48HZqav902X27yK\netPK4q1UXuF7kfPduCxn2z4AtqkhhhJgdfp6Z5IfBh2qqLcz8A6wXTp+H/CDQvx/NeXBRwTZsCx3\nRNJukv4q6XVJ7wCXk/zyrs7rOa83kvyS29y6X8qNI5L/2uXVLSTPGPNaF/BqDfEC/INkZ3KUpK+S\nHGFMzIllb0l/T5st1gNnVBFLVWqMQ9KRkp5Om2TeJjl6yGe5ZcsuX15EfEryfnbNqZPX5yZpJ0mT\nlZwcfwe4NSeOnUkSUlXnSnYG/pNnvJW9EREf5cTQVtJNadPbO8ATlWJYE8mRUgURsQx4BjhW0g4k\n7+GdWxhTZjkRZEPlSxZvBJ4DdomI9sAlJL/QC2kVSXMAAJJExZ1WZVsT4yqSnUeZGi9vTZPS7STN\nQ98CHo6INTlVJgH3ADtHRAfgpjxjqTYOSa1JmqT+C/hCRHQE/paz3NouM11J0oRStrxmJO/vijzi\nquw3wIdA//S9PjUnjmVAD1V99dQy4CuVCyOiNF1em5zinSpXqzR+IdALGJzGcGCl9XSW1L6a+G8j\naR4aCcyIiNerqWfVcCLIpnbAeuC9tK25pvMDdeUhoFjSUWm783kkbduFiHEycL6krumJ35/kMc/t\nJOchTiPZsVSO5a2I+EDSXiTnELY2jm1J2rxXA5+k5xwOypn+BsnOr10Nyx4h6YD0vMCFwAaSZq3N\n1Q54D1gvaWeSZrgy/wLWAr+S1EZSa0n7pNNuAn4saaASvdP5AeYBo5WcMD8CGJpHDBuBdel7VX5J\nb/qrfyowTlJHSS0l7Zcz773AEOBs0hP8tnmcCLLph8C3SXYcN5KcQC6oiHiD5BfbNSQ7lq8Az5L8\ncqzrGG8gab9eAMwk+eVdW3xLSJoYtiU5OZrrLOC/lFx19VPSk6VbE0dEvA1cAEwhOT9xPEmyLJv+\nHMlRyNL0qqAdK8X7PMn7cwNJMhkOjIiIj/OMLdelJCfu1wMPpOstW08pyQnY3Ul+mb+WxkpETCQ5\nmriLpGntXmD7dNZzSS5SeBs4IV1uTa4huZhhLfAU8Eil6WUnhF8iSZLn5MT4Hsm5ge7pX9tMSk+w\nmNWrtKlhJXB8RPyzoeOxxk3S5UD3iDi1oWNpjHxEYPVG0vD00H5bkssTPyb5FW62xdKmpDHA+IaO\npbFyIrD6NBR4maQp4+vAMRFRXdOQWa0knUXSXHV/RDzV0PE0Vm4aMjPLOB8RmJllXKPr7Klz587R\ns2fPhg7DzKxRmT179pqIqPKS7UaXCHr27MmsWbMaOgwzs0ZFUrV32LtpyMws45wIzMwyzonAzCzj\nnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wI\nzMwyzonAzCzjCpYIJN0s6U1Jz1UzXZKul7RE0nxJxYWKxczMqlfII4JbgeE1TD8M6J0OY4EbChiL\nZcCECdCzJzRrlvydMKGhIyo8b7O3uU5ERMEGoCfwXDXTbgRG5Yy/CHyxtmUOGjQorHZ33BHRo0eE\nlPy9446Gjqiw7rgjok2bCPhsaNOmaW+3t9nbvDmAWVHdvrq6CXUx1JIIHgKG5oxPA0qqqTsWmAXM\n6t69++ZtfQZl8Z+lR4+K21s29OjR0JEVjrfZ27w5akoEjeJkcUSMj4iSiCjp0qXKZy/XKGuHkj/7\nGWzcWLFs48akvKl67bXNK28KvM21lzcF9bHNDfnw+hXAzjnj3dKyOjVhAowd+9mO8dVX4fTT4T//\ngSOPhBYtKg4tW25aljtIdR1h3cviP0v37slnW1V5U+VtrljeVNXHNjdkIngAOFvSJGAIsD4iVtX1\nSqr6dfzhh3DppcmwuZo3rzpB1JRAaksuWzpvddM6dYI1azaNvXNnmDIFPv10y4ZPPvn8zte6dXLE\n9+mnn21vs2bQrh0MH558bpWHFi2qLt+Seg2xrJ/+FM4/H95//7Ntbt0afvELKC1NfrTkDk3BlVdW\n/GEH0KZNUt5U1cc2K2k6qnuSJgIHAJ2BN4BLgZYAEfE/kgT8keTKoo3AmIiYVdtyS0pKYtasWquV\na9YsaVGryv33J/8wpaXw8cefva481DRta+bdnOV+8knem9ygpOQ9rzw0b151eW3D5sy3ejW8/DJ8\n8AFsuy306JEkxU8+qX4oe29rG0pLq/8eNSaVk8PmDs2abf0ytnZ9b70FK1bARx8ln3OvXvClL1VM\notX93dJpdVVnS+Zv1gzuvDP5Ufvaa8mRwJVXwujRm/vZa3ZElFQ1rWBHBBExqpbpAXy/UOsvU91h\nVY8eMGJEoddedyI+2yHlk2AeeADGjYPXX4eddoLvfQ+OOGLrdsq1zduUfnlWpewzqIukUhd1cut9\n+mlVpxPrfqiv9dS0zh13hF13TcrLtv/DDzd9T6r7W9200tKG/oZVryxB3HgjnHlm3S+/IZuG6kVT\nOZSUPmv6yceAAXDxxYWNKWs29zOwxqesSXJLEkld1qluWv/+hdnuJv+VLjt82trDKjNr+sqOblu2\nbOhI6leTTwSQ7PS94zczq1qjuI/AzMwKx4nAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwy\nzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6J\nwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDM\nLOMKmggkDZf0oqQlki6qYnp3SdMlPStpvqTDCxmPmZltqmCJQFJzYBxwGNAHGCWpT6VqFwOTI2Ig\ncBLw34WKx8zMqlbII4LBwJKIeDkiPgImAUdXqhNA+/R1B2BlAeMxM7MqFDIRdAWW5YwvT8tyXQac\nLGk58DBwTlULkjRW0ixJs1avXl2IWM3MMquhTxaPAm6NiG7A4cBfJG0SU0SMj4iSiCjp0qVLvQdp\nZtaUFTIRrAB2zhnvlpblOh2YDBAR/wJaAZ0LGJOZmVVSyEQwE+gtqZekbUhOBj9Qqc5rwEEAknYn\nSQRu+zEzq0cFSwQRUQqcDTwGLCK5Ouh5SZdLGpFW+yFwpqR5wETg1IiIQsVkZmabalHIhUfEwyQn\ngXPLLsl5vRDYp5AxmJlZzRr6ZLGZmTUwJwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4\nJwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcCM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLOCcC\nM7OMcyIwM8s4JwIzs4xzIjAzyzgnAjOzjHMiMDPLuLwSgaR7JR0hyYnDzKyJyXfH/t/AN4HFkn4t\nadcCxmRmZvUor0QQEVMjYjRQDCwFpkp6StIYSS0LGaCZmRVW3k09kjoBpwJnAM8C15EkhscLEpmZ\nmdWLFvlUkjQF2BX4C3BURKxKJ90laVahgjMzs8LLKxEA10fE9KomRERJHcZjZmb1LN+moT6SOpaN\nSNpe0vcKFJOZmdWjfBPBmRHxdtlIRKwDzqxtJknDJb0oaYmki6qpc6KkhZKel3RnnvGYmVkdybdp\nqLkkRUQASGoObFPTDGmdccAhwHJgpqQHImJhTp3ewP8D9omIdZJ23JKNMDOzLZfvEcGjJCeGD5J0\nEDAxLavJYGBJRLwcER8Bk4CjK9U5ExiXHmEQEW/mH7qZmdWFfI8IfgJ8BzgrHX8cuKmWeboCy3LG\nlwNDKtX5KoCk/wOaA5dFxCYJRtJYYCxA9+7d8wzZzMzykVciiIhPgRvSoa7X3xs4AOgGzJDUP/d8\nRLr+8cB4gJKSkqjjGMzMMi3f+wh6A/8F9AFalZVHxJdrmG0FsHPOeLe0LNdy4OmI+Bh4RdJLJIlh\nZj5xmZnZ1sv3HMEtJEcDpcAw4HbgjlrmmQn0ltRL0jbAScADlercR3I0gKTOJE1FL+cZk5mZ1YF8\nE0HriJgGKCJejYjLgCNqmiEiSoGzgceARcDkiHhe0uWSRqTVHgPWSloITAcujIi1W7IhZma2ZfI9\nWfxh2gX1YklnkzTxtK1tpoh4GHi4UtklOa8D+EE6mJlZA8j3iOA8oA1wLjAIOBn4dqGCMjOz+lPr\nEUF6Y9jIiPgR8C4wpuBRmZlZvan1iCAiPgGG1kMsZmbWAPI9R/CspAeA/wXeKyuMiHsLEpWZmdWb\nfBNBK2AtcGBOWQBOBGZmjVy+dxb7vICZWROV753Ft5AcAVQQEafVeURmZlav8m0aeijndSvgGGBl\n3YdjZmb1Ld+moXtyxyVNBJ4sSERmZlav8r2hrLLegB8iY2bWBOR7jmADFc8RvE7yjAIzM2vk8m0a\nalfoQMzMrGHk1TQk6RhJHXLGO0r6RuHCMjOz+pLvOYJLI2J92Uj6BLFLCxOSmZnVp3wTQVX18r30\n1MzMPsfyTQSzJF0j6SvpcA0wu5CBmZlZ/cg3EZwDfATcBUwCPgC+X6igzMys/uR71dB7wEUFjsXM\nzBpAvlcNPS6pY8749pIeK1xYZmZWX/JtGuqcXikEQESsw3cWm5k1Cfkmgk8ldS8bkdSTKnojNTOz\nxiffS0B/Bjwp6R+AgH2BsQWLyszM6k2+J4sflVRCsvN/FrgPeL+QgZmZWf3It9O5M4DzgG7AXGAv\n4F9UfHSlmZk1QvmeIzgP2BN4NSKGAQOBt2uexczMGoN8E8EHEfEBgKRtI+IFYNfChWVmZvUl35PF\ny9P7CO4DHpe0Dni1cGGZmVl9yfdk8THpy8skTQc6AI8WLCozM6s3m92DaET8oxCBmJlZw9jSZxab\nmVkT4URgZpZxBU0EkoZLelHSEknV9l4q6ThJkd60ZmZm9ahgiUBSc2AccBjQBxglqU8V9dqR3Kfw\ndKFiMTOz6hXyiGAwsCQiXo6Ij0geaHN0FfWuAH5D8rAbMzOrZ4VMBF2BZTnjy9OycpKKgZ0j4q81\nLUjSWEmzJM1avXp13UdqZpZhDXayWFIz4Brgh7XVjYjxEVESESVdunQpfHBmZhlSyESwAtg5Z7xb\nWlamHdAP+LukpSQd2T3gE8ZmZvWrkIlgJtBbUi9J2wAnAQ+UTYyI9RHROSJ6RkRP4N/AiIiYVcCY\nzMyskoIlgogoBc4GHgMWAZMj4nlJl0saUaj1mpnZ5tnsLiY2R0Q8DDxcqeySauoeUMhYzMysar6z\n2Mws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDM\nLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzj\nnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMq6giUDScEkv\nSloi6aIqpv9A0kJJ8yVNk9SjkPGYmdmmCpYIJDUHxgGHAX2AUZL6VKr2LFASEQOAu4HfFioeMzOr\nWiGPCAYDSyLi5Yj4CJgEHJ1bISKmR8TGdPTfQLcCxmNmZlUoZCLoCizLGV+ellXndOCRqiZIGitp\nlqRZq1evrsMQzczsc3GyWNLJQAlwVVXTI2J8RJREREmXLl3qNzgzsyauRQGXvQLYOWe8W1pWgaSD\ngZ8B+0fEh1uyoo8//pjly5fzwQcfbFGg1jS1atWKbt260bJly4YOxexzrZCJYCbQW1IvkgRwEvDN\n3AqSBgI3AsMj4s0tXdHy5ctp164dPXv2RNLWxGxNRESwdu1ali9fTq9evRo6HLPPtYI1DUVEKXA2\n8BiwCJgcEc9LulzSiLTaVUBb4H8lzZX0wJas64MPPqBTp05OAlZOEp06dfJRolkeCnlEQEQ8DDxc\nqeySnNcH19W6nASsMn8nzPLzuThZbGZmDSeTiWDCBOjZE5o1S/5OmLB1y1u7di177LEHe+yxBzvt\ntBNdu3YtH//oo4/yWsaYMWN48cUXa6wzbtw4JmxtsGZmlRS0aejzaMIEGDsWNqa3sb36ajIOMHr0\nli2zU6dOzJ07F4DLLruMtm3b8qMf/ahCnYggImjWrOrce8stt9S6nu9///tbFmADKi0tpUWLzH3N\nzBqVzB0R/OxnnyWBMhs3JuV1bcmSJfTp04fRo0fTt29fVq1axdixYykpKaFv375cfvnl5XWHDh3K\n3LlzKS0tpWPHjlx00UUUFRWx99578+abyQVVF198Mddee215/YsuuojBgwez66678tRTTwHw3nvv\ncdxxx9GnTx+OP/54SkpKypNUrksvvZQ999yTfv368d3vfpeIAOCll17iwAMPpKioiOLiYpYuXQrA\nr371K/r3709RURE/S9+sspgBXn/9dXbZZRcAbrrpJr7xjW8wbNgwvv71r/POO+9w4IEHUlxczIAB\nA3jooYfK47jlllsYMGAARUVFjBkzhvXr1/PlL3+Z0tJSANatW1dh3MzqXuYSwWuvbV751nrhhRe4\n4IILWLhwIV27duXXv/41s2bNYt68eTz++OMsXLhwk3nWr1/P/vvvz7x589h77725+eabq1x2RPDM\nM89w1VVXlSeVP/zhD+y0004sXLiQn//85zz77LNVznveeecxc+ZMFixYwPr163n00UcBGDVqFBdc\ncAHz5s3jqaeeYscdd+TBBx/kkUce4ZlnnmHevHn88Ic/rHW7n332We69916mTZtG69atue+++5gz\nZw5Tp07lggsuAGDevHn85je/4e9//zvz5s3j6quvpkOHDuyzzz7l8UycOJETTjjBRxVmBZS5RNC9\n++aVb62vfOUrlJSUlI9PnDiR4uJiiouLWbRoUZWJoHXr1hx22GEADBo0qPxXeWXHHnvsJnWefPJJ\nTjrpJACKioro27dvlfNOmzaNwYMHU1RUxD/+8Q+ef/551q1bx5o1azjqqKOA5IasNm3aMHXqVE47\n7TRat24NwA477FDrdh966KFsv/32QJKwLrroIgYMGMChhx7KsmXLWLNmDU888QQjR44sX17Z3zPO\nOKO8qeyWW25hzJgxta7PzLZc5hLBlVdCmzYVy9q0ScoLYbvttit/vXjxYq677jqeeOIJ5s+fz/Dh\nw6u8zn2bbbYpf928efNqm0W23XbbWutUZePGjZx99tlMmTKF+fPnc9ppp23R9fYtWrTg008/Bdhk\n/tztvv3221m/fj1z5sxh7ty5dO7cucb17b///rz00ktMnz6dli1bsttuu212bGaWv8wlgtGjYfx4\n6NEDpOTv+PFbfqJ4c7zzzju0a9eO9u3bs2rVKh577LE6X8c+++zD5MmTAViwYEGVRxzvv/8+zZo1\no3PnzmzYsIF77rkHgO23354uXbrw4IMPAsnOfePGjRxyyCHcfPPNvP/++wC89dZbAPTs2ZPZs2cD\ncPfdd1cb0/r169lxxx1p0aIFjz/+OCtWJD2NHHjggdx1113lyyv7C3DyySczevRoHw2Y1YPMJQJI\ndvpLl8KnnyZ/6yMJABQXF9OnTx922203TjnlFPbZZ586X8c555zDihUr6NOnD7/4xS/o06cPHTp0\nqFCnU6dOfPvb36ZPnz4cdthhDBkypHzahAkTuPrqqxkwYABDhw5l9erVHHnkkQwfPpySkhL22GMP\nfv/73wNw4YUXct1111FcXMy6deuqjelb3/oWTz31FP3792fSpEn07t0bSJqufvzjH7Pffvuxxx57\ncOGFF5bPM3r0aNavX8/IkSPr8u0xsyqo7GqRxqKkpCRmzZpVoWzRokXsvvvuDRTR50tpaSmlpaW0\natWKxYsXc+ihh7J48eJGd7J10qRJPPbYY3ldVlsTfzfMEpJmR0RJVdMa197BavXuu+9y0EEHUVpa\nSkRw4403NrokcNZZZzF16tTyK4fMrLAa1x7CatWxY8fydvvG6oYbbmjoEMwyJZPnCMzM7DNOBGZm\nGedEYGaWcU4EZmYZ50RQB4YNG7bJzWHXXnstZ511Vo3ztW3bFoCVK1dy/PHHV1nngAMOoPLlspVd\ne+21bMzpSe/www/n7bffzid0MzMngrowatQoJk2aVKFs0qRJjBo1Kq/5v/SlL9V4Z25tKieChx9+\nmI4dO27x8upbRJR3VWFm9a/JJYLzz4cDDqjb4fzza17n8ccfz1//+tfyh9AsXbqUlStXsu+++5Zf\n119cXEz//v25//77N5l/6dKl9OvXD0i6fzjppJPYfffdOeaYY8q7dYDk+vqyLqwvvfRSAK6//npW\nrlzJsGHDGDZsGJB0/bBmzRoArrnmGvr160e/fv3Ku7BeunQpu+++O2eeeSZ9+/bl0EMPrbCeMg8+\n+CBDhgxh4MCBHHzwwbzxxhtAcq/CmDFj6N+/PwMGDCjvouLRRx+luLiYoqIiDjroICB5PsPvfve7\n8mX269ePpUuXsnTpUnbddVdOOeUU+vXrx7Jly6rcPoCZM2fyta99jaKiIgYPHsyGDRvYb7/9KnSv\nPXToUObNm1fzB2VmVfJ9BHVghx12YPDgwTzyyCMcffTRTJo0iRNPPBFJtGrViilTptC+fXvWrFnD\nXnvtxYgRI6p9nu4NN9xAmzZtWLRoEfPnz6e4uLh82pVXXskOO+zAJ598wkEHHcT8+fM599xzueaa\na5g+fTqdO3eusKzZs2dzyy238PTTTxMRDBkyhP3335/tt9+exYsXM3HiRP70pz9x4okncs8993Dy\nySdXmH/o0KH8+9//RhI33XQTv/3tb7n66qu54oor6NChAwsWLACSZwasXr2aM888kxkzZtCrV68K\n/QZVZ/Hixdx2223stdde1W7fbrvtxsiRI7nrrrvYc889eeedd2jdujWnn346t956K9deey0vvfQS\nH3zwAUVFRZv1uZlZosklgvRHb70rax4qSwR//vOfgaTZ46c//SkzZsygWbNmrFixgjfeeIOddtqp\nyuXMmDGDc889F4ABAwYwYMCA8mmTJ09m/PjxlJaWsmrVKhYuXFhhemVPPvkkxxxzTHlPoMceeyz/\n/Oc/GTFiBL169WKPPfYAqu/qevny5YwcOZJVq1bx0Ucf0atXLwCmTp1aoSls++2358EHH2S//fYr\nr5NPV9U9evQoTwLVbZ8kvvjFL7LnnnsC0L59ewBOOOEErrjiCq666ipuvvlmTj311FrXZ2ZVa3JN\nQw3l6KOPZtq0acyZM4eNGzcyaNAgIOnEbfXq1cyePZu5c+fyhS98YYu6fH7llVf43e9+x7Rp05g/\nfz5HHHHEFi2nTFkX1lB9N9bnnHMOZ599NgsWLODGG2/c6q6qoWJ31bldVW/u9rVp04ZDDjmE+++/\nn8mTJzO6vnoONGuCnAjqSNu2bRk2bBinnXZahZPEZV0wt2zZkunTp/Pqq6/WuJz99tuPO++8E4Dn\nnnuO+fPnA0kX1ttttx0dOnTgjTfe4JFHHimfp127dmzYsGGTZe27777cd999bNy4kffee48pU6aw\n77775r1N69evp2vXrgDcdttt5eWHHHII48aNKx9ft24de+21FzNmzOCVV14BKnZVPWfOHADmzJlT\nPr2y6rZv1113ZdWqVcycOROADRs2lCetM844g3PPPZc999yz/CE4Zrb5nAjq0KhRo5g3b16FRDB6\n9GhmzZpF//79uf3222t9yP9SnTEAAAbVSURBVMpZZ53Fu+++y+67784ll1xSfmRRVFTEwIED2W23\n3fjmN79ZoQvrsWPHMnz48PKTxWWKi4s59dRTGTx4MEOGDOGMM85g4MCBeW/PZZddxgknnMCgQYMq\nnH+4+OKLWbduHf369aOoqIjp06fTpUsXxo8fz7HHHktRUVF599HHHXccb731Fn379uWPf/wjX/3q\nV6tcV3Xbt80223DXXXdxzjnnUFRUxCGHHFJ+pDBo0CDat2/vZxaYbSV3Q22N1sqVKznggAN44YUX\naNas6t80/m6YJWrqhtpHBNYo3X777QwZMoQrr7yy2iRgZvlpclcNWTaccsopnHLKKQ0dhlmT0GR+\nSjW2Ji4rPH8nzPLTJBJBq1atWLt2rf/xrVxEsHbtWlq1atXQoZh97jWJpqFu3bqxfPlyVq9e3dCh\n2OdIq1at6NatW0OHYfa51yQSQcuWLcvvaDUzs81T0KYhScMlvShpiaSLqpi+raS70ulPS+pZyHjM\nzGxTBUsEkpoD44DDgD7AKEl9KlU7HVgXEbsAvwd+U6h4zMysaoU8IhgMLImIlyPiI2AScHSlOkcD\nZX0X3A0cpOq65TQzs4Io5DmCrsCynPHlwJDq6kREqaT1QCdgTW4lSWOBsenou5Je3MKYOldedgZ4\nm7PB25wNW7PNPaqb0ChOFkfEeGD81i5H0qzqbrFuqrzN2eBtzoZCbXMhm4ZWADvnjHdLy6qsI6kF\n0AFYW8CYzMyskkImgplAb0m9JG0DnAQ8UKnOA8C309fHA0+E7wozM6tXBWsaStv8zwYeA5oDN0fE\n85IuB2ZFxAPAn4G/SFoCvEWSLAppq5uXGiFvczZ4m7OhINvc6LqhNjOzutUk+hoyM7Mt50RgZpZx\nmUgEkm6W9Kak5xo6lvoiaWdJ0yUtlPS8pPMaOqZCk9RK0jOS5qXb/IuGjqk+SGou6VlJDzV0LPVB\n0lJJCyTNlTSr9jkaP0kdJd0t6QVJiyTtXafLz8I5Akn7Ae8Ct0dEv4aOpz5I+iLwxYiYI6kdMBv4\nRkQsbODQCia9K327iHhXUkvgSeC8iPh3A4dWUJJ+AJQA7SPiyIaOp9AkLQVKIiIzN5NJug34Z0Tc\nlF6F2SYi3q6r5WfiiCAiZpBclZQZEbEqIuakrzcAi0ju5G6yIvFuOtoyHZr0Lx1J3YAjgJsaOhYr\nDEkdgP1IrrIkIj6qyyQAGUkEWZf26joQeLphIym8tJlkLvAm8HhENPVtvhb4MfBpQwdSjwL4m6TZ\nafczTV0vYDVwS9oEeJOk7epyBU4ETZyktsA9wPkR8U5Dx1NoEfFJROxBcif7YElNtilQ0pHAmxEx\nu6FjqWdDI6KYpGfj76dNv01ZC6AYuCEiBgLvAZt06781nAiasLSd/B5gQkTc29Dx1Kf00Hk6MLyh\nYymgfYARaZv5JOBASXc0bEiFFxEr0r9vAlNIejpuypYDy3OObu8mSQx1xomgiUpPnP4ZWBQR1zR0\nPPVBUhdJHdPXrYFDgBcaNqrCiYj/FxHdIqInyV35T0TEyQ0cVkFJ2i69+IG0eeRQoElfDRgRrwPL\nJO2aFh0E1OlFH42i99GtJWkicADQWdJy4NKI+HPDRlVw+wDfAhakbeYAP42IhxswpkL7InBb+lCk\nZsDkiMjEJZUZ8gVgSvrYkhbAnRHxaMOGVC/OASakVwy9DIypy4Vn4vJRMzOrnpuGzMwyzonAzCzj\nnAjMzDLOicDMLOOcCMzMMs6JwCwl6ZO0R8uyoc7u3pTUM0u931rjkon7CMzy9H7aPYVZpviIwKwW\naf/3v037wH9G0i5peU9JT0iaL2mapO5p+RckTUmfizBP0tfSRTWX9Kf0WQl/S+9+RtK56XMj5kua\n1ECbaRnmRGD2mdaVmoZG5kxbHxH9gT+S9PgJ8AfgtogYAEwArk/Lrwf+ERFFJH3CPJ+W9wbGRURf\n4G3guLT8ImBgupzvFmrjzKrjO4vNUpLejYi2VZQvBQ6MiJfTjvxej4hOktaQPPzn47R8VUR0lrQa\n6BYRH+YsoydJt9i90/GfAC0j4peSHiV5cNJ9wH05z1Qwqxc+IjDLT1TzenN8mPP6Ez47R3cEMI7k\n6GGmJJ+7s3rlRGCWn5E5f/+Vvn6KpNdPgNHAP9PX04CzoPxBOR2qW6ikZsDOETEd+AnQAdjkqMSs\nkPzLw+wzrXN6agV4NCLKLiHdXtJ8kl/1o9Kyc0ieGnUhyROkynqEPA8YL+l0kl/+ZwGrqllnc+CO\nNFkIuL6uH0NoVhufIzCrRRYflm7Z4qYhM7OM8xGBmVnG+YjAzCzjnAjMzDLOicDMLOOcCMzMMs6J\nwMws4/4/VFh3y7a61vwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZxU5Z3v8c+XRRFBiIAS2ZprjIgL\ni33JEBU1Jg5ucFVUkKjRGDJe10S9IcY40cQxTjLKaLhOCIkxAzYSjcs4OiZR4jLGpVFAFo1c0moj\nIgFFAbfW3/3jHNqiqd6gTxfd5/t+vXhVnec89dTvVPOqb53nnDqliMDMzPKrQ6kLMDOz0nIQmJnl\nnIPAzCznHARmZjnnIDAzyzkHgZlZzjkIrMVI6ihpg6SBLdm3lCR9TlIm51jXHVvS7yVNzqIOSd+X\n9G/b+nhr3xwEOZa+EW/+94mk9wqWi74hNSQiPo6IbhHxakv23VFJ+qOkq4q0nyxppaSOzRkvIo6O\niNktUNeXJVXVGfuHEfEP2zt2I88Zki5trJa0/QlJXytYHiLpTklrJa2XtFDSJZL8HtUK/CLnWPpG\n3C0iugGvAicUtG31hiSpU+tXuUO7DTijSPsZwKyI+LiV6ymls4B1wJnNfaCkfYCngBXAARHRA5gI\njAa6tmSRVpyDwOol6UeS7pBUIeld4KuSRkt6StLbklZJuklS57R/p/RTYVm6PCtd/6CkdyX9WdLg\n5vZN1x8j6S/pp8WbJf134SfKOnU3pcZvSlou6S1JNxU8tqOkG9NPpiuAsQ28RL8D+kr6YsHjewHH\nAr9Jl8dJWiDpHUmvSvp+A6937afkxuqQdK6kZelr9f8knZu29wD+AxhYsHe3R/q3/HXB40+UtCR9\njR6RtG/BumpJ35b0Qvp6V0jauYG6uwMnAf8bGCppeAOvWTE/BB6NiP8TEasAImJZRJwWERuaOZZt\nAweBNeZE4HagB3AHUANcDPQGDiF5g/pmA48/Hfg+sDvJXscPm9tX0h7AXODy9Hn/CoxqYJym1Hgs\ncDAwgiTgvpy2nwccDQwD/idwan1PEhEbgTvZ8lPwRGBRRCxJlzcAk4GewAnAxZKOb6D2zRqrYzVw\nHLAb8A3gZkkHRcT69HleLdi7e7PwgZL2A/4duBDoA/wRuG9zWKZOBb4C/A+S16nYns9mE4C3gN+m\nY53VhO0r9GWS19FKxEFgjXkiIv4jIj6JiPci4tmIeDoiaiJiBTADOLyBx98ZEZUR8REwG2jo02J9\nfY8HFkTEvem6G4G/1TdIE2u8LiLWR0QV8KeC5zoVuDEiqiNiLfDjBuqFZHro1IJPzGembZtreSQi\nlqSv30JgTpFaimmwjvRvsiISjwAPA4c1YVxIwuq+tLaP0rF7AF8o6DMtIt5In/t+Gv67nQXMiYhP\nSD40nN7MacTdgVXN6G8tzEFgjXmtcCE9qPefkt6Q9A5wDckn7/q8UXB/E9BtG/ruVVhHJFdKrK5v\nkCbW2KTnAl5poF6AR4F3gBMkfZ5kD6OioJbRkv4kaY2k9cC5RWoppsE6JB0v6WlJ6yS9TbL30JRx\nN49dO176Bl4N9Cvo06S/Wzq1N4YkuAHuTvtunsqqATpv9cCk7aP0/jrgs02s3TLgILDG1D1l8efA\nYuBzEbEbcBWgjGtYBfTfvCBJbPmmVdf21LgKGFCw3ODprWko/YZkT+AM4IGIKNxbmQPcBQxID4LO\nbGIt9dYhaReSqZTrgD0joifw+4JxGzvN9HVgUMF4HUhe35VNqKuuM9PnfVDSG8ByYCc+nR56FdhD\nUu1B3/TvN4hPw+iPwMnb8NzWQhwE1lzdgfXAxnSuuaHjAy3lfmCkpBPSKYeLSea2s6hxLnCJpH7p\ngd/vNOExvyH5BHwOBdNCBbWsi4j3Jf0dybTM9taxM8mb7Rrg4/SYw1EF61cDvdODuPWNPU7SEelx\ngcuBd4Gnm1hboTNJgnZ4wb/TSPaQPkNyPOd54MeSdpXUBZgKbASeTce4CjhC0nWS+gJI+ryk2yU1\ntAdpLcRBYM11KcmnvXdJPnnfkfUTRsRqkjeXG4C1wN4kby4fZFDjLSTz7S+QvFE1ehAzIpYDz5C8\nQf9nndXnAdcpOevqCpI34e2qIyLeBr5FMg2zjuRg7f0F6xeT7IVUpWcF7VGn3iUkr88tJGEyFhiX\nHi9oMkmHkkwzTU+PJ7wREW+kdVUBp6V7TBNI9uBWkExBHQ4cFxEfpPX8heRU0c8DS9Oprrkkp5Ru\nak5Ntm3kH6axtkbJF7VeByZExOOlrsesrfMegbUJksZK6pmenfN9kgONz5S4LLN2wUFgbcWhJFML\na4C/B07cPLVgZtvHU0NmZjnnPQIzs5xrcxcR6927d5SVlZW6DDOzNmX+/Pl/i4iip123uSAoKyuj\nsrKy1GWYmbUpkur9lrynhszMcs5BYGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnO\nOQjMzHLOQWBmlnMOAjOznHMQmJnlnIPAzCznHARmZjnnIDAzy7nMgkDSryS9KWlxPesl6SZJyyUt\nkjQyq1ryaPZsKCuDDh2S29mzS11R9rzN3ub2KvNtjohM/gFjgJHA4nrWHws8CAj4O+Dppox78MEH\nR3PNmhUxaFCElNzOmtXsIdqUWbMiunaNgE//de3avrfb2+xtbq9aapuByqjv/bq+FS3xDyhrIAh+\nDkwqWH4J+GxjYzY3CPL4H2fQoC23d/O/QYNKXVl2vM3e5vaqpba5oSAo5U9V9gNeK1iuTttW1e0o\naQowBWDgwIHNepLvfQ82bdqybdMmuPRSGDAAOnZMdrfqu21oXVP6dugAUvNemO316qvNa28PvM2N\nt7cH3ubG27dFm/jN4oiYAcwAKC8vj+Y8tr4Xa/VqOPzw7S6tSaSWC5am9Nl5Z3j//a3r6NIFjj9+\n6/Yo8orWbWtKn1I+bqed4IMPtl6/885w1FHJ36DwH7St5WJt3brBu+9uvc3dusG3v71l/yzra82x\nP/MZWLdu623efXf4zW8+/eBV321D67b1NosxC8fu3x9ee23rbW7mZ+IGlTIIVgIDCpb7p20tauBA\neKXITzbvuWdywOWTT+Djj4vfNrRuW/pmPV5NTXLbvz+sWJG0byYl27xqVfE9lKa07ciPGzwY/vKX\nLbe5Q4fk7//BB1vuVEPbWm6oTzEffQQzZ7bM87cVa9fCWWeVuorW07UrXHtty41XyiC4D7hA0hzg\nC8D6iNhqWmh7XXstTJmy5fRQ167wL/+SfFJsr2bPTqbFXn01eTO89lqYPLnUVWXL25zdNjcUFFmF\nYH3r7roLrr8eVq6Efv3gsstg/PjkQ0BE47dN6bOtt1mNWVkJDz4Ib78Ngwa1/N9ZkVHsS6oAjgB6\nA6uBfwQ6A0TEv0kS8DNgLLAJODsiKhsbt7y8PCorG+22hTy+QZiZFZI0PyLKi67LKgiysi1BYGaW\ndw0Fgb9ZbGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnOOQjMzHLOQWBmlnMOAjOz\nnHMQmJnlnIPAzCznHARmZjnnIDAzyzkHgZlZzjkIzMxyzkFgZpZzDgIzs5xzEJiZ5ZyDwMws5xwE\nZmY55yAwM8s5B4GZWc45CMzMcs5BYGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnO\nZRoEksZKeknScklTi6wfKGmepOclLZJ0bJb1mJnZ1jILAkkdgenAMcBQYJKkoXW6XQnMjYgRwETg\n/2ZVj5mZFZflHsEoYHlErIiID4E5wPg6fQLYLb3fA3g9w3rMzKyILIOgH/BawXJ12lboB8BXJVUD\nDwAXFhtI0hRJlZIq16xZk0WtZma5VeqDxZOAX0dEf+BY4N8lbVVTRMyIiPKIKO/Tp0+rF2lm1p5l\nGQQrgQEFy/3TtkJfB+YCRMSfgS5A7wxrMjOzOrIMgmeBfSQNlrQTycHg++r0eRU4CkDSfiRB4Lkf\nM7NWlFkQREQNcAHwELCM5OygJZKukTQu7XYp8A1JC4EK4GsREVnVZGZmW+uU5eAR8QDJQeDCtqsK\n7i8FDsmyBjMza1ipDxabmVmJOQjMzHLOQWBmlnMOAjOznHMQmJnlnIPAzCznHARmZjnnIDAzyzkH\ngZlZzjkIzMxyzkFgZpZzDgIzs5xzEJiZ5ZyDwMws5xwEZmY55yAwM8s5B4GZWc45CMzMcs5BYGaW\ncw4CM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnOOQjMzHLOQWBmlnMOAjOznHMQmJnlnIPA\nzCznMg0CSWMlvSRpuaSp9fQ5VdJSSUsk3Z5lPWZmtrVOWQ0sqSMwHfgKUA08K+m+iFha0Gcf4LvA\nIRHxlqQ9sqrHzMyKy3KPYBSwPCJWRMSHwBxgfJ0+3wCmR8RbABHxZob1mJlZEVkGQT/gtYLl6rSt\n0OeBz0v6b0lPSRpbbCBJUyRVSqpcs2ZNRuWameVTqQ8WdwL2AY4AJgG/kNSzbqeImBER5RFR3qdP\nn1Yu0cysfas3CCT9vaQJRdonSPpKE8ZeCQwoWO6fthWqBu6LiI8i4q/AX0iCwczMWklDewRXAY8W\naf8TcE0Txn4W2EfSYEk7AROB++r0uYdkbwBJvUmmilY0YWwzM2shDQXBzhGx1YR8RPwN2LWxgSOi\nBrgAeAhYBsyNiCWSrpE0Lu32ELBW0lJgHnB5RKxt7kaYmdm2a+j00d0kdUrf0GtJ6gzs0pTBI+IB\n4IE6bVcV3A/g2+k/MzMrgYb2CH5HcvC29tO/pG7Av6XrzMysHWgoCK4EVgOvSJov6Tngr8CadJ2Z\nmbUD9U4NpVNCUyVdDXwubV4eEe+1SmVmZtYq6g0CSSfVaQqgp6QFEfFutmWZmVlraehg8QlF2nYH\nDpL09Yh4JKOazMysFTU0NXR2sXZJg4C5wBeyKsrMzFpPsy8xERGvAJ0zqMXMzEqg2UEgaQjwQQa1\nmJlZCTR0sPg/SA4QF9od+Czw1SyLMjOz1tPQweKf1lkOYB1JGHwV+HNWRZmZWetp6GBx7QXnJI0A\nTgdOIflS2V3Zl2ZmZq2hoamhz5P8RsAk4G/AHYAi4shWqs3MzFpBQ1NDLwKPA8dHxHIASd9qlarM\nzKzVNHTW0EnAKmCepF9IOgpQ65RlZmatpd4giIh7ImIiMITktwIuAfaQdIuko1urQDMzy1aj3yOI\niI0RcXtEnEDyc5PPA9/JvDIzM2sVzfpCWUS8lf6Q/FFZFWRmZq2r2d8sNjOz9sVBYGaWcw4CM7Oc\ncxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnOOQjMzHLOQWBmlnMOAjOznHMQmJnlnIPAzCznHARm\nZjmXaRBIGivpJUnLJU1toN/JkkJSeZb1mJnZ1jILAkkdgenAMcBQYJKkoUX6dQcuBp7OqhYzM6tf\nlnsEo4DlEbEiIj4E5gDji/T7IXA98H6GtZiZWT2yDIJ+wGsFy9VpWy1JI4EBEfGfDQ0kaYqkSkmV\na9asaflKzcxyrGQHiyV1AG4ALm2sb/rzmOURUd6nT5/sizMzy5Esg2AlMKBguX/atll34ADgT5Kq\ngL8D7vMBYzOz1pVlEDwL7CNpsKSdgInAfZtXRsT6iOgdEWURUQY8BYyLiMoMazIzszoyC4KIqAEu\nAB4ClgFzI2KJpGskjcvqec3MrHk6ZTl4RDwAPFCn7ap6+h6RZS1mZlacv1lsZpZzDgIzs5xzEJiZ\n5ZyDwMws5xwEZmY55yAwM8s5B4GZWc45CMzMcs5BYGaWcw4CM7OccxCYmeWcg8DMLOccBGZmOecg\nMDPLOQeBmVnOOQjMzHLOQWBmlnMOAjOznHMQmJnlnIPAzCznHARmZjnnIDAzyzkHgZlZzjkIzMxy\nzkFgZpZzDgIzs5xzEJiZ5ZyDwMws5xwEZmY55yAwM8u5TINA0lhJL0laLmlqkfXflrRU0iJJD0sa\nlGU9Zma2tcyCQFJHYDpwDDAUmCRpaJ1uzwPlEXEQcCfwz1nVY2ZmxWW5RzAKWB4RKyLiQ2AOML6w\nQ0TMi4hN6eJTQP8M6zEzsyKyDIJ+wGsFy9VpW32+DjxYbIWkKZIqJVWuWbOmBUs0M7Md4mCxpK8C\n5cBPiq2PiBkRUR4R5X369Gnd4szM2rlOGY69EhhQsNw/bduCpC8D3wMOj4gPtuWJPvroI6qrq3n/\n/fe3qVArjS5dutC/f386d+5c6lLMci3LIHgW2EfSYJIAmAicXthB0gjg58DYiHhzW5+ourqa7t27\nU1ZWhqTtqdlaSUSwdu1aqqurGTx4cKnLMcu1zKaGIqIGuAB4CFgGzI2IJZKukTQu7fYToBvwW0kL\nJN23Lc/1/vvv06tXL4dAGyKJXr16eS/ObAeQ5R4BEfEA8ECdtqsK7n+5pZ7LIdD2+G9mtmPYIQ4W\nm5lZ6eQyCGbPhrIy6NAhuZ09e/vGW7t2LcOHD2f48OH07duXfv361S5/+OGHTRrj7LPP5qWXXmqw\nz/Tp05m9vcUWWL16NZ06dWLmzJm1bTU1NfTs2XOLfjNnzuSSSy6pXf71r3/NAQccwIEHHsjIkSO5\n8cYbW6wmM2t9mU4N7Yhmz4YpU2BT+jW2V15JlgEmT962MXv16sWCBQsA+MEPfkC3bt247LLLtugT\nEUQEHToUz95bb7210ec5//zzt63AesydO5fRo0dTUVHBueee26TH3H///fzsZz/jj3/8I3379uX9\n999n1qxZLVqXmbWu3O0RfO97n4bAZps2Je0tbfny5QwdOpTJkyez//77s2rVKqZMmUJ5eTn7778/\n11xzTW3fQw89lAULFtR+Ip86dSrDhg1j9OjRvPlmckLVlVdeybRp02r7T506lVGjRrHvvvvy5JNP\nArBx40ZOPvlkhg4dyoQJEygvL68NqboqKiqYNm0aK1asYNWqVU3apn/6p3/ihhtuoG/fvkByCmhT\nQ8TMdky5C4JXX21e+/Z68cUX+da3vsXSpUvp168fP/7xj6msrGThwoX84Q9/YOnSpVs9Zv369Rx+\n+OEsXLiQ0aNH86tf/aro2BHBM888w09+8pPaULn55pvp27cvS5cu5fvf/z7PP/980cdWVVWxbt06\nDj74YE455RTmzp3bpO1ZsmQJBx98cBO33szagtwFwcCBzWvfXnvvvTfl5eW1yxUVFYwcOZKRI0ey\nbNmyokGwyy67cMwxxwBw8MEHU1VVVXTsk046aas+TzzxBBMnTgRg2LBh7L///kUfO2fOHE477TQA\nJk6cSEVFBVD/mTw+w8es/cpdEFx7LXTtumVb165JexZ23XXX2vsvv/wy//qv/8ojjzzCokWLGDt2\nbNHz6Hfaaafa+x07dqSmpqbo2DvvvHOjfepTUVHBzJkzKSsr46STTuK5555jxYoVdOzYkQ4dOmwx\n3rp16+jduzcAQ4cOZf78+c16LjPbseUuCCZPhhkzYNAgkJLbGTO2/UBxc7zzzjt0796d3XbbjVWr\nVvHQQw+1+HMccsghtdM8L7zwQtE9jqVLl1JTU8PKlSupqqqiqqqKyy+/nDlz5gAwZswYbr/9dgA2\nbdrEb3/7W4488kgAvvvd73LZZZexevVqAD744AN++ctftvh2mFnryd1ZQ5C86bfGG39dI0eOZOjQ\noQwZMoRBgwZxyCGHtPhzXHjhhZx55pkMHTq09l+PHj226FNRUcGJJ564RdvJJ5/MWWedxRVXXMHN\nN9/MN7/5TW644QY++eQTzjnnHL74xS8CMG7cONasWcOXvvQlIJky+sY3vtHi22FmrUcRUeoamqW8\nvDwqKyu3aFu2bBn77bdfiSrasdTU1FBTU0OXLl14+eWXOfroo3n55Zfp1GnHzHz/7cxah6T5EVFe\nbN2O+e5g22zDhg0cddRR1NTUEBH8/Oc/32FDwMx2DH6HaGd69uzpg7lm1iy5O1hsZmZbchCYmeWc\ng8DMLOccBGZmOecgaAFHHnnkVl8OmzZtGuedd16Dj+vWrRsAr7/+OhMmTCja54gjjqDu6bJ1TZs2\njU0FV9I79thjefvtt5tSepMMHz689rIV9dVVVVXFAQccULv8zDPPMGbMGPbdd19GjBjBueeeu0WN\nZrbjcBC0gEmTJtV+K3ezOXPmMGnSpCY9fq+99uLOO+/c5uevGwQPPPDAVr8psK2WLVvGxx9/zOOP\nP87GjRub9JjVq1dzyimncP311/PSSy/x/PPPM3bsWN59990WqcnMWla7O330kkugnqsub7PhwyG9\n+nNREyZM4Morr+TDDz9kp512oqqqitdff53DDjuMDRs2MH78eN566y0++ugjfvSjHzF+/PgtHl9V\nVcXxxx/P4sWLee+99zj77LNZuHAhQ4YM4b333qvtd9555/Hss8/y3nvvMWHCBK6++mpuuukmXn/9\ndY488kh69+7NvHnzKCsro7Kykt69e3PDDTfUXr303HPP5ZJLLqGqqopjjjmGQw89lCeffJJ+/fpx\n7733sssuu2y1bRUVFZxxxhksW7aMe++9l9NPP73R12v69OmcddZZjB49eovXyMx2TN4jaAG77747\no0aN4sEHHwSSvYFTTz0VSXTp0oW7776b5557jnnz5nHppZfS0Le5b7nlFrp27cqyZcu4+uqrt/hO\nwLXXXktlZSWLFi3i0UcfZdGiRVx00UXstddezJs3j3nz5m0x1vz587n11lt5+umneeqpp/jFL35R\ne1nql19+mfPPP58lS5bQs2dP7rrrrqL13HHHHUycOJFJkybVXqG0MYsXL/alqs3akHa3R9DQJ/cs\nbZ4eGj9+PHPmzKm9EFtEcMUVV/DYY4/RoUMHVq5cyerVq2t/2KWuxx57jIsuugiAgw46iIMOOqh2\n3dy5c5kxYwY1NTWsWrWKpUuXbrG+rieeeIITTzyx9gqoJ510Eo8//jjjxo1j8ODBDB8+HKj/Uteb\n9yoGDhxIv379OOecc1i3bh2777570ctS+1LVZm2T9whayPjx43n44Yd57rnn2LRpU+0n4tmzZ7Nm\nzRrmz5/PggUL2HPPPYteeroxf/3rX/npT3/Kww8/zKJFizjuuOO2aZzNNl/CGuq/jHVFRQUvvvgi\nZWVl7L333rzzzju1ew69evXirbfequ1beKnq/fff399uNmtDHAQtpFu3bhx55JGcc845WxwkXr9+\nPXvssQedO3dm3rx5vPLKKw2OU3gJ6MWLF7No0SIguYT1rrvuSo8ePVi9enXtNBRA9+7dix6IPeyw\nw7jnnnvYtGkTGzdu5O677+awww5r0vZ88sknzJ07lxdeeKH2UtX33ntv7fTQEUccwaxZs2qnuW67\n7bbaS1VfcMEF3HbbbTz99NO14/3ud7+rvXS1me1Y2t3UUClNmjSJE088cYsziCZPnswJJ5zAgQce\nSHl5OUOGDGlwjPPOO4+zzz6b/fbbj/322692z2LYsGGMGDGCIUOGMGDAgC0uYT1lyhTGjh1be6xg\ns5EjR/K1r32NUaNGAcnB4hEjRtT7i2eFHn/8cfr168dee+1V2zZmzBiWLl1a+9vLL774IsOGDUMS\n5eXlXHfddQDsueeezJkzh8suu4w333yTDh06MGbMGMaOHdv4i2hmrc6XobaS8t/OrHU0dBlqTw2Z\nmeWcg8DMLOfaTRC0tSku89/MbEfRLoKgS5curF271m8sbUhEsHbtWrp06VLqUsxyr12cNdS/f3+q\nq6tZs2ZNqUuxZujSpQv9+/cvdRlmudcugqBz584MHjy41GWYmbVJmU4NSRor6SVJyyVNLbJ+Z0l3\npOufllSWZT1mZra1zIJAUkdgOnAMMBSYJGlonW5fB96KiM8BNwLXZ1WPmZkVl+UewShgeUSsiIgP\ngTnA+Dp9xgO3pffvBI6Sr1xmZtaqsjxG0A94rWC5GvhCfX0iokbSeqAX8LfCTpKmAFPSxQ2SXtrG\nmnrXHTsHvM354G3Oh+3Z5kH1rWgTB4sjYgYwY3vHkVRZ31es2ytvcz54m/Mhq23OcmpoJTCgYLl/\n2la0j6ROQA9gbYY1mZlZHVkGwbPAPpIGS9oJmAjcV6fPfcBZ6f0JwCPhb4WZmbWqzKaG0jn/C4CH\ngI7AryJiiaRrgMqIuA/4JfDvkpYD60jCIkvbPb3UBnmb88HbnA+ZbHObuwy1mZm1rHZxrSEzM9t2\nDgIzs5zLRRBI+pWkNyUtLnUtrUXSAEnzJC2VtETSxaWuKWuSukh6RtLCdJuvLnVNrUFSR0nPS7q/\n1LW0BklVkl6QtEBSZeOPaPsk9ZR0p6QXJS2TNLpFx8/DMQJJY4ANwG8i4oBS19MaJH0W+GxEPCep\nOzAf+F8RsbTEpWUm/Vb6rhGxQVJn4Ang4oh4qsSlZUrSt4FyYLeIOL7U9WRNUhVQHhG5+TKZpNuA\nxyNiZnoWZteIeLulxs/FHkFEPEZyVlJuRMSqiHguvf8usIzkm9ztViQ2pIud03/t+pOOpP7AccDM\nUtdi2ZDUAxhDcpYlEfFhS4YA5CQI8i69qusI4OnSVpK9dJpkAfAm8IeIaO/bPA34P8AnpS6kFQXw\ne0nz08vPtHeDgTXArekU4ExJu7bkEzgI2jlJ3YC7gEsi4p1S15O1iPg4IoaTfJN9lKR2OxUo6Xjg\nzYiYX+paWtmhETGS5MrG56dTv+1ZJ2AkcEtEjAA2Altd1n97OAjasXSe/C5gdkT8rtT1tKZ013ke\nMLbUtWToEGBcOmc+B/iSpFmlLSl7EbEyvX0TuJvkSsftWTVQXbB3eydJMLQYB0E7lR44/SWwLCJu\nKHU9rUFSH0k90/u7AF8BXixtVdmJiO9GRP+IKCP5Vv4jEfHVEpeVKUm7pic/kE6PHA2067MBI+IN\n4DVJ+6ZNRwEtetJHm7j66PaSVAEcAfSWVA38Y0T8srRVZe4Q4AzghXTOHOCKiHighDVl7bPAbemP\nInUA5kZELk6pzJE9gbvTny3pBNweEf9V2pJaxYXA7PSMoRXA2S05eC5OHzUzs/p5asjMLOccBGZm\nOecgMDPLOQeBmVnOOQjMzHLOQWCWkvRxekXLzf9a7NubksrydPVba1ty8T0CsyZ6L708hVmueI/A\nrBHp9e//Ob0G/jOSPpe2l+66P70AAAGOSURBVEl6RNIiSQ9LGpi27ynp7vR3ERZK+mI6VEdJv0h/\nK+H36befkXRR+rsRiyTNKdFmWo45CMw+tUudqaHTCtatj4gDgZ+RXPET4Gbgtog4CJgN3JS23wQ8\nGhHDSK4JsyRt3weYHhH7A28DJ6ftU4ER6Tj/kNXGmdXH3yw2S0naEBHdirRXAV+KiBXphfzeiIhe\nkv5G8uM/H6XtqyKit6Q1QP+I+KBgjDKSy2Lvky5/B+gcET+S9F8kP5x0D3BPwW8qmLUK7xGYNU3U\nc785Pii4/zGfHqM7DphOsvfwrCQfu7NW5SAwa5rTCm7/nN5/kuSqnwCTgcfT+w8D50HtD+X0qG9Q\nSR2AARExD/gO0APYaq/ELEv+5GH2qV0KrtQK8F8RsfkU0s9IWkTyqX5S2nYhya9GXU7yC1Kbrwh5\nMTBD0tdJPvmfB6yq5zk7ArPSsBBwU0v/DKFZY3yMwKwRefyxdMsXTw2ZmeWc9wjMzHLOewRmZjnn\nIDAzyzkHgZlZzjkIzMxyzkFgZpZz/x8hyQqx8MY2NQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXwV9b3/8dcnbCEQdqjKFtQUiEpY\nIgjiQrX+sLXgQlXE9mr10nJd6fKTbla9tfXeWhesDyuKXrymIGqptMUdrPpTgYCAGFQWEQIIAVlk\nEwKf3x8zSU7CyUZyckjm/Xw8ziPnzEzmfOYE5n3m+535jrk7IiISXSnJLkBERJJLQSAiEnEKAhGR\niFMQiIhEnIJARCTiFAQiIhGnIJAaM7MmZrbbzHrU5bLJZGYnm1lCzqUuv24ze8XMxiWiDjP7tZn9\n+Wh//1hjZm+b2TXh8+vN7I3kVtQ4KQgiINwRFz8Om9m+mNdxd0iVcfdD7t7a3dfV5bLHKjN7zcxu\njzP9MjPbYGZNarI+d7/A3XProK7zzWxtuXX/p7v/qLbrjvNe15vZofDfzC4ze9/MLqzr95HkUBBE\nQLgjbu3urYF1wHdiph2xQzKzpvVf5TFtGvC9ONO/Bzzt7ofquZ5keSv8N9QOeByYaWbpSa5J6oCC\nQDCz35rZM2Y23cy+BK42s6Fm9p6Z7TCzTWY22cyahcs3NTM3s4zw9dPh/BfN7Esze9fMetV02XD+\nhWb2iZntNLOHzOz/FTcNxKm7OjX+0MxWmdl2M5sc87tNzOx+M9tmZmuAkZV8RH8FjjOzYTG/3xH4\nFvBU+HqUmS0Jvy2vM7NfV/J5xzZ3VFpH+E18RfhZrTaz68PpbYG/Az1iju66hH/L/4n5/UvM7MPw\nM5prZr1j5hWY2Y/N7IPw855uZi0q+RwAcPfDwP8CrYGTY9Z3ZszfY4mZnR37eZnZ/4R/p+1m9nzM\n9DlmVhhO/7uZda2qBqlbCgIpdgnwF6At8AxQBNwCdALOJNhB/bCS378K+DXQgeCo4z9ruqyZdQFm\nAj8L3/dTYHAl66lOjd8CBgEDCALu/HD6BOACIBs4Hbi8ojdx9z3Ac8D3YyZfCSxz9w/D17uBcQTf\nlr8D3GJmF1VSe7Gq6tgMfBtoA/w78JCZ9XP3neH7rIs5utsS+4tm1pdgh30T0Bl4DZhdHJahy4Fv\nAicSfE7xjnzKCI8YrwUOAOvDad2B2cBvCP6uk4C/hoEJwb+t5kAW0AV4MJyeAjwG9AB6Agdj5kk9\nURBIsbfd/e/uftjd97n7Qnef7+5F7r4GmAKcU8nvP+fuee5+EMgF+h/FshcBS9z9hXDe/cDWilZS\nzRp/7+473X0t8EbMe10O3O/uBe6+DbinknohaB66POYb8/fDacW1zHX3D8PPbykwI04t8VRaR/g3\nWeOBucDrwFnVWC8EYTU7rO1guO62wJCYZR5w98/D9/4Hlf/dhpvZDmAf8HvgKncv/vt8P3yvl8PP\n4CVgKTAyDInzgAnuvt3dD7r7m+H2Fbr7rPDf3C7gd1Tvc5M6pCCQYutjX5hZHzP7p5l9bma7gLsI\nvnlX5POY53sJmg1quuwJsXV4MCJiQUUrqWaN1Xov4LNK6gX4F7AL+I6ZfZ3gCGN6TC1DzeyNsIlj\nJ3B9nFriqbQOM7vIzOab2RfhTviCaq63eN0l6wubdAqA2KaXmvzd3nb3dgTf+OcAw2Pm9QTGhs1C\nO8Jazwhr6A5sDY9iyjCz1mb2eNictguYW4PtkzqiIJBi5U9ZfBRYDpzs7m2A2wFLcA2bgG7FL8zM\nKLvTKq82NW4i2EEVq/T01jCUniL45vs9YE7Mt2EIjgCeB7q7e1uCztTq1FJhHWbWkqBJ6vfA18Kd\n8Csx663qNNONBDvo4vWlEHy+G6pRV4Xc/UuCJq3rzKxfOHk98KS7t4t5tHL3P4TzOplZmzir+xnQ\nCxgc/g2/UZva5OgoCKQi6cBOYE/Y1lxZ/0Bd+Qcw0My+E7ZD30LQtp2IGmcCt5pZ17Ad+7Zq/M5T\nBP0QPyCmWSimli/cfb+ZnUHQLFPbOloQtKsXAofCPofzYuZvJtjBVnTmzkxglJmdG/YL/Az4Ephf\nzdoq5O6FwBMEfT0Q9EVcYmbfDDvAU81shJmd4O7rCfonHjazdmbWLKYjOZ3gSGR7uP1HnKYriacg\nkIr8BPg3gh3HowQdyAnl7puBK4D7gG3AScD7wFcJqPERgvb2D4CFBN+8q6pvFbCAYAf9z3KzJwC/\nt+Csq18Q7IRrVYe77wAmArOAL4AxBGFZPH85wVHI2rA5pku5ej8k+HweIQiTkcCosL+gLtxPEDRZ\nYR/MJQTBUEhwEsBPKN3HXB3+/IQgwG4KX99H0G+xDXgHeLGOapMaMN2YRo5VFlyotREY4+5vJbse\nkcZKRwRyTDGzkWHzQQuCb5cHCb6Fi0iCKAjkWDMcWEPQvPB/gEvcvaKmIRGpA2oaEhGJOB0RiIhE\nXIMbXKxTp06ekZGR7DJERBqURYsWbXX3uKdjN7ggyMjIIC8vL9lliIg0KGZW4dXzahoSEYk4BYGI\nSMQpCEREIk5BICIScQoCEZGIUxCIiEScgkBEJOIUBCIiEacgEBGJOAWBiEjEKQhERCJOQSAiEnEK\nAhGRiFMQiIhEnIJARCTiEhYEZvaEmW0xs+UVzDczm2xmq8xsmZkNTFQtEg25uZCRASkpwc/c3GRX\nlHjaZm1znXD3hDyAs4GBwPIK5n8LeBEw4AxgfnXWO2jQIJeqPf20e8+e7mbBz6efTnZFifX00+5p\nae5Q+khLa9zbrW3WNtcEkOcV7FcTevN6M8sA/uHup8aZ9yjwhrtPD19/DJzr7psqW2dOTo7rDmWV\ny82F8eNh797SaWlpMGUKjBuXvLoSKSMDPotz/6WePWHt2vqupn5om0tpm6tmZovcPSfevGT2EXQF\n1se8LginHcHMxptZnpnlFRYW1ktxDdkvf1k2BCB4/ctfJqee+rBuXc2mNwba5qqnNwb1sc0NorPY\n3ae4e46753TuHPfey5WKWptiFP+z9OhRs+mNgba56umNQX1sczKDYAPQPeZ1t3BanSpuJvnss6B1\n7bPPgteNOQyi+J/l7ruD5q9YaWnB9MZK2xzQNteBijoP6uIBZFBxZ/G3KdtZvKA666xpZ3HPnmU7\nWYofPXvWaDUNShQ71Nyj10Hurm3WNlcfyegsNrPpwLlAJ2Az8BugWRg+fzYzA/4EjAT2Ate6e5W9\nwDXtLE5JCXaFR9YHhw9XezUNTm5u0Cewbl1wJHD33Y23o1hEqlZZZ3FCzxpKhJoGQRTPMhARKe9Y\nPWuoXkSxTVFEpCYafRCMGxecP9+zZ9Ac1LNn4z6fXkSkppomu4D6MG6cdvwiIhVp9EcEIiJSOQWB\niEjEKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTiFAQiIhGnIBARiTgFgYhIxCkIREQiTkEgIhJx\nCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYm4yATBjh2wfXuyqxAROfZEJgiefBI6dICvfx2uvhom\nT4b33oP9+5NdmYhIckXinsUA558Pv/89LFgAb7wBubnB9KZNoV8/GDy49NGnDzRpktRyRUTqjbl7\nsmuokZycHM/Ly6v1ejZsgIULg8eCBcHPnTuDea1bw6BBpcFw+unQoweY1fptRUSSwswWuXtOvHmR\nOSIor2vX4HHxxcHrw4dh5crSYFiwAB58EA4cCOZ36VI2GE4/HTp2TF79IiJ1JbJBUF5KCvTuHTyu\nvjqYduAALFtWNhz++U8oPog66aTSYBg8GAYMgLS05G2DiMjRiGzT0NHatQsWLSobDuvXB/OaNIHT\nTisNhsGDISsr6IcQEUmmypqGFAR14PPPywbDggXB6aoQHCEMHFi2MzojQ/0NIlK/FAT1zB1WrSob\nDosXw1dfBfM7diwbDKefDp07J7fmhu7AAdi2DQoLoVkzOPFEaNEi2VWJHDvUWVzPzCAzM3hcdVUw\n7eBBWL687FHDyy8HndQQHCXEBsPAgcHZS1HkDrt3w9atwY69sLDq58VnfBUzg549S/8OmZnBNSSZ\nmcFn3axZUjZN5JikI4Ik2r07OFKIDYfPPgvmpaTAKaeUDYdTT22YO7BDh+CLL2q2Yy8+eiqvefPg\n6KlTp+Bn+eedOwcXCa5cGTw++ST4GRsUTZsGYVAcDLFB0b27riGRxilpTUNmNhJ4EGgCPO7u95Sb\n3wOYBrQLl5nk7nMqW2djCoJ4tmw5sr/hiy+Ceamppf0NxR3SJ51U//0N+/dXb2de/HzbttIzrcpr\n06byHXv55+npNd9e96CO2GCIfezZU7ps8+bBZ1r+KCIzE044IQhokYYoKUFgZk2AT4BvAgXAQmCs\nu+fHLDMFeN/dHzGzLGCOu2dUtt7GHgTlucOnn5YNhsWLYd++YH779mWD4fTT4bjjarb+nTurt1Mv\nfr17d/x1paSU7rirs2Pv2DH57fjusGlT2WAoDotVq8oembRsWfYIIjYsunTRCQBybEtWH8FgYJW7\nrwmLmAGMBvJjlnGgTfi8LbAxgfU0SGZBx+eJJ8KVVwbTiorgww/LhsPvflfa39C9e2mTUs+epZ2o\n8XbyW7cG/RfxtGxZdufdu3flO/j27RveN2az4Jv+CSfAOeeUnXf4MBQUHHkUsXw5vPBC8Hcolp4e\n/ygiM1MXHsqxL5FHBGOAke5+ffj6e8AQd78xZpnjgVeA9kAr4Hx3XxRnXeOB8QA9evQY9FlxQ7qU\n2LMH3n+/NBgWLoQ1a8ou0759zZphWrVKzrY0BEVFQX9O+aOIlSth7drSUIZgsMN4ndaZmUHTmEh9\nSFbTUHWC4MdhDX80s6HAVOBUdz8cd6VEr2moNrZuDa5x6NQp+FbaEDuaG6IDB4LmvPJHEp98Unrx\nYbEuXY48gsjMhJNPVhBL3UpW09AGoHvM627htFjXASMB3P1dM0sFOgFbElhXZHTqFDykfjVvXjpc\nSXn79sHq1UceRbz0UjBUeqyuXeMfRZx0UvL7VqRxSWQQLAQyzawXQQBcCVxVbpl1wHnA/5hZXyAV\nKExgTSJJ1bJlcBrwqaceOe/LL4MO6vJHEbNmBUd3xcyC0XBjw+H444PrTlq3DvorYn+2atXw+m6k\nfiUsCNy9yMxuBF4mODX0CXf/0MzuAvLcfTbwE+AxM5tI0HF8jTe0CxtE6kh6ejBw4YABR87bsSP+\n6a+5uUdeTBdPq1ZlwyFeYNRkWmqqzpJqTHRBmUgDVnyNxNatwRHF7t2lP2OfV2fal1+W7eSuTJMm\nNQuR6oSMBmdMLA0xIdJImZWe6VVb7sHFgkcTIsXP168vOy32Yr2qtGhR/aOS9PTg1Oi+fYM+E4VI\n7ejjExEgCJWWLUuvH6kLhw8HYVCbI5VNm8pOKz/8SPPmQT9J377BsO99+waP3r2DJiypmoJARBIm\nJSX49p6eHnRo14UDB4JgWLMGVqwIHvn5sHQp/PWvpc1bKSnQq1fZcMjKCu5Jrus3ylIQiEiD0rx5\ncF1Mx47BkCqxigccLA6H4qB4+eXS285CcGpu+YDo2ze6w8ErCESk0UhNDe4SeNppZacXFQUX+cWG\nQ34+TJ1ath+jY8fSUIgNim7dGvdZUjprSEQiyz0YT6p8QKxYEYzRVSw9PWhSKh8QJ57YcIYt1x3K\nRERqqLCwNBRig2JDzPgILVoEF/aVD4ivf/3Yu/pbp4+KiNRQ587BiLTlR6XduRM++qhsQOTlwbPP\nlt53IyUlOK01Xkf1sXjnQQWBiEgNtG0LQ4YEj1j79gVXfpdvZpozp+xQ7927x++oTuZw5QoCEZE6\n0LIlZGcHj1gHDwYDDZbvg3jrLdi7t3S5zp3jB8QJJyS+o1pBICKSQM2aBU1CffrAJZeUTj98GNat\nOzIgZswIxpYq1qZNaThcey2cfXbd16ggEBFJgpQUyMgIHhdeWDrdHTZvPrKT+qWX4BvfSEwtCgIR\nkWOIWXDf8eOOgxEjys5L1EmeGqVcRKSBSFRfgYJARCTiFAQiIhGnIBARiTgFgYhIxCkIREQiTkEg\nIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYk4BYGISMQpCEREIk5BICIScQoCEZGIUxCIiERc\nQoPAzEaa2cdmtsrMJlWwzOVmlm9mH5rZXxJZj4iIHKnSW1Wa2Y8rm+/u91Xyu02Ah4FvAgXAQjOb\n7e75MctkAj8HznT37WbWpSbFi4hI7VV1z+L0Wqx7MLDK3dcAmNkMYDSQH7PMvwMPu/t2AHffUov3\nExGRo1BpELj7nbVYd1dgfczrAmBIuWW+DmBm/w9oAtzh7i+VX5GZjQfGA/To0aMWJYmISHlVNQ1N\nrmy+u99cB++fCZwLdAPeNLPT3H1HufeZAkwByMnJ8Vq+p4iIxKiqaWhRLda9Aege87pbOC1WATDf\n3Q8Cn5rZJwTBsLAW7ysiIjVQVdPQtFqseyGQaWa9CALgSuCqcsv8DRgLPGlmnQiaitbU4j1FRKSG\nqjoiAMDMOgO3AVlAavF0d/9GRb/j7kVmdiPwMkH7/xPu/qGZ3QXkufvscN4FZpYPHAJ+5u7bjnpr\nRESkxqoVBEAu8AzwbeBHwL8BhVX9krvPAeaUm3Z7zHMHfhw+REQkCap7QVlHd58KHHT3f7n7D4AK\njwZERKThqO4RwcHw5yYz+zawEeiQmJJERKQ+VTcIfmtmbYGfAA8BbYCJCatKRETqTbWCwN3/ET7d\nCYxIXDkiIlLfqtVHYGbTzKxdzOv2ZvZE4soSEZH6Ut3O4n6xV/uGYwMNSExJIiJSn6obBClm1r74\nhZl1oPr9CyIicgyr7s78j8C7ZvZs+Pq7wN2JKUlEROpTdTuLnzKzPEqvHbg09r4CIiLScNXkDmUd\ngD3u/iegMBxDSEREGrjqnjX0G4Kxhn4eTmoGPJ2ookREpP5U94jgEmAUsAfA3TdSu7uXiYjIMaK6\nQXAgHCDOAcysVeJKEhGR+lTdIJhpZo8C7czs34HXgMcTV5aIiNSX6p41dK+ZfRPYBfQGbnf3VxNa\nmYiI1ItqXxQW7vhfBTCzFDMb5+65CatMRETqRaVNQ2bWxsx+bmZ/MrMLLHAjwe0kL6+fEkVEJJGq\nOiL4X2A78C5wPfALwICL3X1JgmsTEZF6UFUQnOjupwGY2ePAJqCHu+9PeGUiIlIvqjprqPjOZLj7\nIaBAISAi0rhUdUSQbWa7wucGtAxfG8G959sktDoREUm4SoPA3ZvUVyEiIpIcNRl0TkREGiEFgYhI\nxCkIREQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYm4hAaBmY00s4/NbJWZTapkucvM\nzM0sJ5H1iIjIkRIWBGbWBHgYuBDIAsaaWVac5dKBW4D5iapFREQqlsgjgsHAKndf4+4HgBnA6DjL\n/SfwX4BGNRURSYJEBkFXYH3M64JwWgkzGwh0d/d/VrYiMxtvZnlmlldYWFj3lYqIRFjSOovNLAW4\nD/hJVcu6+xR3z3H3nM6dOye+OBGRCElkEGwAuse87hZOK5YOnAq8YWZrgTOA2eowFhGpX4kMgoVA\nppn1MrPmwJXA7OKZ7r7T3Tu5e4a7ZwDvAaPcPS+BNYmISDkJCwJ3LwJuBF4GVgAz3f1DM7vLzEYl\n6n1FRKRmqrpVZa24+xxgTrlpt1ew7LmJrEVEROLTlcUiIhGnIBARiTgFgYhIxCkIREQiTkEgIhJx\nCgIRkYhTEIiIRJyCQEQk4hQEIiIRpyAQEYk4BYGISMQpCEREIk5BICIScQoCEZGIUxCIiEScgkBE\nJOIUBCIiEacgEBGJOAWBiEjEKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTiFAQiIhGnIBARiTgF\ngYhIxCkIREQiTkEgIhJxCgIRkYhLaBCY2Ugz+9jMVpnZpDjzf2xm+Wa2zMxeN7OeiaxHRESOlLAg\nMLMmwMPAhUAWMNbMssot9j6Q4+79gOeA/05UPSIiEl8ijwgGA6vcfY27HwBmAKNjF3D3ee6+N3z5\nHtAtgfWIiEgciQyCrsD6mNcF4bSKXAe8GG+GmY03szwzyyssLKzDEkVE5JjoLDazq4Ec4A/x5rv7\nFHfPcfeczp07129xIiKNXNMErnsD0D3mdbdwWhlmdj7wS+Acd//qaN7o4MGDFBQUsH///qMqVI5d\nqampdOvWjWbNmiW7FJFGK5FBsBDINLNeBAFwJXBV7AJmNgB4FBjp7luO9o0KCgpIT08nIyMDM6tN\nzXIMcXe2bdtGQUEBvXr1SnY5Io1WwpqG3L0IuBF4GVgBzHT3D83sLjMbFS72B6A18KyZLTGz2Ufz\nXvv376djx44KgUbGzOjYsaOO9EQSLJFHBLj7HGBOuWm3xzw/v67eSyHQOOnvKpJ4x0RnsYiIJE8k\ngyA3FzIyICUl+JmbW7v1bdu2jf79+9O/f3+OO+44unbtWvL6wIED1VrHtddey8cff1zpMg8//DC5\ntS02NHz4cHr37k12djaDBw9m2bJldbLeYqtWraJ///4AvPbaa1x88cV1un4RqTsJbRo6FuXmwvjx\nsDe8jO2zz4LXAOPGHd06O3bsyJIlSwC44447aN26NT/96U/LLOPuuDspKfGz98knn6zyfW644Yaj\nK7ACzzzzDP379+exxx7jtttu48UX417GISKNXOSOCH75y9IQKLZ3bzC9rq1atYqsrCzGjRvHKaec\nwqZNmxg/fjw5OTmccsop3HXXXSXLDh8+nCVLllBUVES7du2YNGkS2dnZDB06lC1bghOqfvWrX/HA\nAw+ULD9p0iQGDx5M7969eeeddwDYs2cPl112GVlZWYwZM4acnJySkKrI0KFD2bCh9MzeF198kaFD\nhzJw4ECuuOIK9uzZA8D8+fMZOnQo2dnZDBkyhL1797J69WrOOussBgwYwKBBg5g/f36dfoYikniR\nC4J162o2vbY++ugjJk6cSH5+Pl27duWee+4hLy+PpUuX8uqrr5Kfn3/E7+zcuZNzzjmHpUuXMnTo\nUJ544om463Z3FixYwB/+8IeSUHnooYc47rjjyM/P59e//jXvv/9+lTW+9NJLJU03W7Zs4Z577uH1\n119n8eLF9OvXjwcffJD9+/dz5ZVX8vDDD7N06VJeeeUVWrRowfHHH8+rr77K+++/T25uLjfffHMt\nPi0RSYbINQ316BE0B8WbnggnnXQSOTk5Ja+nT5/O1KlTKSoqYuPGjeTn55OVVXYsvpYtW3LhhRcC\nMGjQIN56662467700ktLllm7di0Ab7/9NrfddhsA2dnZnHLKKRXWdsUVV/DVV1+xb9++kqOGd955\nh/z8fIYNGwbAgQMHGD58OCtWrKBHjx4MHDgQgLZt2wLw1VdfceONN7J06VKaNm3K6tWra/T5iEjy\nRe6I4O67IS2t7LS0tGB6IrRq1ark+cqVK3nwwQeZO3cuy5YtY+TIkXHPkW/evHnJ8yZNmlBUVBR3\n3S1atKhymco888wzfPrpp4wdO5ZbbrkFCI4yRo4cyZIlS1iyZAn5+flMmTKlwnX88Y9/pHv37nzw\nwQcsWLCAr746qovDRSSJIhcE48bBlCnQsyeYBT+nTDn6juKa2LVrF+np6bRp04ZNmzbx8ssv1/l7\nnHnmmcycOROADz74IG7TUywz43e/+x1vvvkmK1euZNiwYfzrX/9izZo1QNDnsHLlSrKysli3bh2L\nFy8u2ZZDhw6xc+dOjj/+eMyMadOm4e51vk0ikliRCwIIdvpr18Lhw8HP+ggBgIEDB5KVlUWfPn34\n/ve/z5lnnlnn73HTTTexYcMGsrKyuPPOO8nKyippxqlIWloaEydO5N577+VrX/saU6dO5YorriA7\nO5thw4bxySef0KJFC6ZPn86ECRPIzs7mggsuKGkWevzxx8nOzubTTz8tOUoRkYbDGto3uJycHM/L\nyyszbcWKFfTt2zdJFR1bioqKKCoqIjU1lZUrV3LBBRewcuVKmjZtuN1B+vuK1J6ZLXL3nHjzGu7e\nQeLavXs35513HkVFRbg7jz76aIMOARFJPO0hGpl27dqxaNGiZJchIg1IJPsIRESklIJARCTiFAQi\nIhGnIBARiTgFQR0YMWLEEReHPfDAA0yYMKHS32vdujUAGzduZMyYMXGXOffccyl/umx5DzzwAHtj\nRtL71re+xY4dO6pTeqXuuOOOkiG1s7KymD59eq3XWV5GRgZbt24FSj8PEalfCoI6MHbsWGbMmFFm\n2owZMxg7dmy1fv+EE07gueeeO+r3Lx8Ec+bMoV27dke9vlgTJ05kyZIlvPDCC/zwhz/k4MGDdbJe\nETl2NLrTR2+9FaoYdbnG+veHcPTnuMaMGcOvfvUrDhw4QPPmzVm7di0bN27krLPOYvfu3YwePZrt\n27dz8OBBfvvb3zJ69Ogyv7927Vouuugili9fzr59+7j22mtZunQpffr0Yd++fSXLTZgwgYULF7Jv\n3z7GjBnDnXfeyeTJk9m4cSMjRoygU6dOzJs3j4yMDPLy8ujUqRP33Xdfyeil119/Pbfeeitr167l\nwgsvZPjw4bzzzjt07dqVF154gZYtW1a4jZmZmaSlpbF9+3a6dOnC6tWrueGGGygsLCQtLY3HHnuM\nPn36sHnzZn70ox+VDFHxyCOPMGzYMC6++GLWr1/P/v37ueWWWxhffBMIEUm6RhcEydChQwcGDx7M\niy++yOjRo5kxYwaXX345ZkZqaiqzZs2iTZs2bN26lTPOOINRo0ZVeC/eRx55hLS0NFasWMGyZctK\nRvsEuPvuu+nQoQOHDh3ivPPOY9myZdx8883cd999zJs3j06dOpVZ16JFi3jyySeZP38+7s6QIUM4\n55xzaN++PStXrmT69Ok89thjXH755Tz//PNcffXVFW7j4sWLyczMpEuXLgCMHz+eP//5z2RmZjJ/\n/nz+4z/+g7lz53LzzTdzzjnnMGvWLA4dOsTu3bsBeOKJJ+jQoQP79u3j9NNP57LLLqNjx461/ehF\npA40uiCo7Jt7IhU3DxUHwdSpU4FgNM9f/OIXvPnmm6SkpLBhwwY2b97McccdF3c9b775ZsmY/v36\n9aNfv34l82bOnMmUKVMoKjM9vSkAAAhVSURBVCpi06ZN5Ofnl5lf3ttvv80ll1xSMgLqpZdeyltv\nvcWoUaPo1atXya0kY4exLu/+++/nySef5JNPPuHvf/87EFy9/M477/Dd7363ZLniUUfnzp3LU089\nBQSjohaPczR58mRmzZoFwPr161m5cqWCQOQY0eiCIFlGjx7NxIkTWbx4MXv37mXQoEEA5ObmUlhY\nyKJFi2jWrBkZGRlxh56uyqeffsq9997LwoULad++Pddcc81RradY7OBwTZo0KdMEFWvixIn89Kc/\nZfbs2Vx33XWsXr2aw4cP065duyrvfFbsjTfe4LXXXuPdd98lLS2Nc889t1a1i0jdUmdxHWndujUj\nRozgBz/4QZlO4p07d9KlSxeaNWvGvHnz+CzeXXFinH322fzlL38BYPny5SU3ld+1axetWrWibdu2\nbN68ucz9hdPT0/nyyy+PWNdZZ53F3/72N/bu3cuePXuYNWsWZ5111lFt36hRo8jJyWHatGm0adOG\nXr168eyzzwLBUc/SpUsBOO+883jkkUcASoap3rlzJ+3btyctLY2PPvqI995776hqEJHEUBDUobFj\nx7J06dIyQTBu3Djy8vI47bTTeOqpp+jTp0+l65gwYQK7d++mb9++3H777SVHFtnZ2QwYMIA+ffpw\n1VVXlRnCevz48YwcOZIRI0aUWdfAgQO55pprGDx4MEOGDOH6669nwIABR719t99+O/fddx+HDx8m\nNzeXqVOnltwF7YUXXgDgwQcfZN68eZx22mkMGjSI/Px8Ro4cSVFREX379mXSpEmcccYZR12DiNQ9\nDUMtxzz9fUVqr7JhqHVEICIScQoCEZGIazRB0NCauKR69HcVSbxGEQSpqals27ZNO41Gxt3Ztm0b\nqampyS5FpFFrFNcRdOvWjYKCAgoLC5NditSx1NRUunXrluwyRBq1RhEEzZo1o1evXskuQ0SkQUpo\n05CZjTSzj81slZlNijO/hZk9E86fb2YZiaxHRESOlLAgMLMmwMPAhUAWMNbMssotdh2w3d1PBu4H\n/itR9YiISHyJPCIYDKxy9zXufgCYAYwut8xoYFr4/DngPKtoWE4REUmIRPYRdAXWx7wuAIZUtIy7\nF5nZTqAjsDV2ITMbDxQPYL/bzD4+ypo6lV93BGibo0HbHA212eaeFc1oEJ3F7j4FmFLb9ZhZXkWX\nWDdW2uZo0DZHQ6K2OZFNQxuA7jGvu4XT4i5jZk2BtsC2BNYkIiLlJDIIFgKZZtbLzJoDVwKzyy0z\nG/i38PkYYK7rqjARkXqVsKahsM3/RuBloAnwhLt/aGZ3AXnuPhuYCvyvma0CviAIi0SqdfNSA6Rt\njgZtczQkZJsb3DDUIiJStxrFWEMiInL0FAQiIhEXiSAwsyfMbIuZLU92LfXFzLqb2TwzyzezD83s\nlmTXlGhmlmpmC8xsabjNdya7pvpgZk3M7H0z+0eya6kPZrbWzD4wsyVmllf1bzR8ZtbOzJ4zs4/M\nbIWZDa3T9Uehj8DMzgZ2A0+5+6nJrqc+mNnxwPHuvtjM0oFFwMXunp/k0hImvCq9lbvvNrNmwNvA\nLe7+XpJLSygz+zGQA7Rx94uSXU+imdlaIMfdI3MxmZlNA95y98fDszDT3H1HXa0/EkcE7v4mwVlJ\nkeHum9x9cfj8S2AFwZXcjZYHdocvm4WPRv1Nx8y6Ad8GHk92LZIYZtYWOJvgLEvc/UBdhgBEJAii\nLhzVdQAwP7mVJF7YTLIE2AK86u6NfZsfAP4vcDjZhdQjB14xs0Xh8DONXS+gEHgybAJ83Mxa1eUb\nKAgaOTNrDTwP3Oruu5JdT6K5+yF3709wJftgM2u0TYFmdhGwxd0XJbuWejbc3QcSjGx8Q9j025g1\nBQYCj7j7AGAPcMSw/rWhIGjEwnby54Fcd/9rsuupT+Gh8zxgZLJrSaAzgVFhm/kM4Btm9nRyS0o8\nd98Q/twCzCIY6bgxKwAKYo5unyMIhjqjIGikwo7TqcAKd78v2fXUBzPrbGbtwuctgW8CHyW3qsRx\n95+7ezd3zyC4Kn+uu1+d5LISysxahSc/EDaPXAA06rMB3f1zYL2Z9Q4nnQfU6UkfDWL00doys+nA\nuUAnMysAfuPuU5NbVcKdCXwP+CBsMwf4hbvPSWJNiXY8MC28KVIKMNPdI3FKZYR8DZgV3rakKfAX\nd38puSXVi5uA3PCMoTXAtXW58kicPioiIhVT05CISMQpCEREIk5BICIScQoCEZGIUxCIiEScgkAk\nZGaHwhEtix91dvWmmWVEafRbaVgicR2BSDXtC4enEIkUHRGIVCEc//6/wzHwF5jZyeH0DDOba2bL\nzOx1M+sRTv+amc0K74uw1MyGhatqYmaPhfdKeCW8+hkzuzm8b8QyM5uRpM2UCFMQiJRqWa5p6IqY\neTvd/TTgTwQjfgI8BExz935ALjA5nD4Z+Je7ZxOMCfNhOD0TeNjdTwF2AJeF0ycBA8L1/ChRGydS\nEV1ZLBIys93u3jrO9LXAN9x9TTiQ3+fu3tHMthLc/OdgOH2Tu3cys0Kgm7t/FbOODIJhsTPD17cB\nzdz9t2b2EsGNk/4G/C3mngoi9UJHBCLV4xU8r4mvYp4forSP7tvAwwRHDwvNTH13Uq8UBCLVc0XM\nz3fD5+8QjPoJMA54K3z+OjABSm6U07ailZpZCtDd3ecBtwFtgSOOSkQSSd88REq1jBmpFeAldy8+\nhbS9mS0j+FY/Npx2E8Fdo35GcAep4hEhbwGmmNl1BN/8JwCbKnjPJsDTYVgYMLmub0MoUhX1EYhU\nIYo3S5doUdOQiEjE6YhARCTidEQgIhJxCgIRkYhTEIiIRJyCQEQk4hQEIiIR9/8Bzbvo2grbzWwA\nAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZgU5bn38e+PRQFBUcGjAWQwKrIv\nDqhH3BU1ieKuHIzBjcgbF8xJIq8eo/HE15zEo0YjKm7RSESiosQlGo1GPYnIoICKIhyCyqICBhcW\nceB+/6iasWeYpYHpaZn6fa6rru566umn7urp6bvrqaqnFBGYmVl2NSt2AGZmVlxOBGZmGedEYGaW\ncU4EZmYZ50RgZpZxTgRmZhnnRGBVSGou6XNJuzZk3WKStLukgpwnXb1tSU9LGlGIOCRdLunWTX19\nsUn6nqQn86h3h6RLGyMmS8jXEWzZJH2eM9sG+AJYl85/PyImNH5UXy+SdgfmRoRqWPYM8EJEXFWt\n/ETgRmDXiFhX/XX5tL2ZdQ8H7oiIkvrqbi5J5wC3AauB9cD/ApdFxBOFXrd9PXiPYAsXEW0rJuA9\n4Jicsg2SgKQWjR/l19o9wHdrKP8ucF9dSaCJeTH9DG0P3Av8QdJ21Sv589M0ORE0cZJ+LukBSfdL\n+gw4XdJ+kl6WtELSEkk3SmqZ1m8hKSSVpPP3pcuflPSZpL9L6raxddPlR0t6R9Inkm6S9D+SRtYS\ndz4xfl/SPEn/lHRjzmubS7pe0nJJ84Gj6niLHgZ2lvSvOa/fEfgWyRciko6VNEPSp5Lek3R5He/3\nSxXbVF8cks6R9Fb6Xv1v+suc9Av4j8Cuadfb55J2Sv+Wv815/fGS3kzfo79I6p6zbKGkH0p6PX2/\n75e0dR3vAwBp4ruLZO+ym6TDJS2QdKmkD4Dbc96Tmem6X5LUO2fdXSU9ImmppGWSfp2zvc+nz5ul\nf9OP0vhmSeqZLrtP0pU57Z2X/p2Xp+3ukpbX+Tmw/DkRZMPxwO+B7YAHgHLgIqADsD/JF9T363j9\nvwGXAzuQ7HX858bWlbQTMAn4cbrefwCD62gnnxi/BewNDCBJcIen5aOBoUA/YBBwSm0riYiVwIPA\nGTnFpwGzIuLNdP5zYATQHjgGuEjSd+qIvUJ9cXwIfBvYFjgXuElS34j4JF3Pezl7dx/lvlBSD+B3\nwAVAR+AZYEpFskydAhwB7EbyPtW051OFkl/8ZwOfkXQRAXQG2gK7Av9H0iCShHAOsCNJ4nhU0lbp\n6x8H5gElQBeSv3t1RwP7AnuQ7IWcBnxcQzxDgauAk4BOwGKg+p5ubZ8Dy5MTQTa8FBF/jIj1EbE6\nIqZFxNSIKI+I+cB44KA6Xv9gRJRFxJck/4T9N6Hud4AZEfFouux6YFltjeQZ4zUR8UlELACez1nX\nKcD1EbEwIpYDv6gjXki6h07J+cV8RlpWEctfIuLN9P2bCUysIZaa1BlH+jeZH4m/AM8CB+TRLiRf\nnFPS2L5M294O2Cenzg0R8UG67seo++82RNIK4AOSL93jIuKzdFk5cGVErI2I1cAoYFz6N1oXEXel\n9QYB+5Ek70siYmX6efufGtb3JUkC3Ct9L2ZHxAc11BtBcqxkRkSsAcYCB0nqnFOnts+B5cmJIBve\nz52RtJekxyV9IOlTkl9cHep4fe4/6CqSX4cbW/cbuXFEcpbCwtoayTPGvNYFvFtHvAB/BT4FjpG0\nJ8kvy/tzYtlP0vNpV8cnJL+E63q/KtQZh6TvSJoq6eP0S3honu1WtF3ZXkSsJ3k/O+XU2Zi/20sR\n0T4iOkTEv6aJqcKHEbE2Z74rcEnaLbQijX2XdN1dgAX1HVuJiKeBW4FbgA8l3SqpXR7b+Snwz83Y\nTquBE0E2VD817DbgDWD3iNgW+ClQ75ksm2kJSRcDAJJE1X/m6jYnxiUkX0gV6jy9NU1K95LsCXwX\neCIicvdWJgIPAV0iYjvgjjxjqTUOSa1JuqSuAf4lItoDT+e0W9/pfItJvpAr2mtG8v4uyiOujVU9\nlveBn6WJo2JqExGT0mVdJTWvt9GIGyJiINAb6An8sIZq1bezHUlXUiG2M7OcCLKpHfAJsDLta67r\n+EBDeQwYKOmYtB/5IpK+7ULEOAkYI6lTeuD3kjxecy/JcYizyOkWyonl44hYI2lfkm6ZzY1ja2Ar\nYCmwLj3mcFjO8g+BDrX8Sq5o+1hJB6fHBX5M0q8/Nc/YNsftwA8kDVKibfp33Qb4O7Ac+H+S2khq\nLWn/6g1IGpxOLYCVwFqSU1erux84W1LftOvuGpIznGrdm7SN50SQTf8OfI/ki+M2kgPIBRURHwKn\nAteRfFF8E3iN5LqHho7xFpL+9teBaSS/vOuLbx7wCskX9OPVFo8GrlFy1tWl1Hzwc6PiiIgVwMXA\nZJKDpCeRJMuK5W+Q7IUsSLtfdqoW75sk788tJMnkKODY9HhBQUXEyyTvyS0k3TTvAKeny8pJjgf1\nINk7eI9k26prD9wJrAAWkOw9XVfDuv5E0i04Oa2zK8lxA2tAvqDMiiLtOlgMnBQRLxY7HrMs8x6B\nNRpJR0lqn+7iX05y5sgrRQ7LLPOcCKwxDQHmk3RlHAkcHxG1dQ2ZWSNx15CZWcZ5j8DMLOO2uAGk\nOnToECUlJcUOw8xsizJ9+vRlEVHjKdtbXCIoKSmhrKys2GGYmW1RJNV6hb27hszMMs6JwMws45wI\nzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzM\nMs6JwMws45wIzMwyrmCJQNJdkj6S9EYtyyXpRknzJM2SNLBQsZiZWe0KuUfwW+CoOpYfDeyRTqOA\nWwoYi1mTNGEClJRAs2bJ44QJxY6o8LzNDb/NBUsEEfEC8HEdVYYB90biZaC9pF0KFU/W+J+l6W/z\nhAkwahS8+y5EJI+jRjXt7fY2F2ibI6JgE1ACvFHLsseAITnzzwKltdQdBZQBZbvuumtY3e67L6JN\nm4jkY5NMbdok5U1VFre5a9eq21sxde1a7MgKx9u86dsMlEUt39VbxMHiiBgfEaURUdqxY433Xq5T\n1n4pXnYZrFpVtWzVqqS8qcriNr/33saVNwXe5vrLN0UxE8EioEvOfOe0rEFlcVfS/yz1lzcFu+66\nceVNgbe5/vJNUcxEMAU4Iz17aF/gk4hY0tAryeIvRf+z1F/eFFx9NbRpU7WsTZukvKnyNicafJtr\n6zPa3Am4H1gCfAksBM4GzgPOS5cLuBn4X+B1ajk+UH3ae++9N6pfTKq5f03aqGa2KFnsL8/iNkck\n29e1a/J57tq16W9vhLd5U7eZOo4RKFm+5SgtLY2ysrK865eUJN1B1XXtCgsWNFhYXzsTJiR7Pe+9\nl/wqvvpqGDGi2FEVVha32SxfkqZHRGmNy5p6Iqg4RpDbPdSmDYwf7y8JM8uOuhLBFnHW0OYYMSL5\n0u/aFaTk0UnAzOwrLYodQGMYMcJf/GZmtWnyewRmZlY3JwIzs4zLRNeQbdki4MsvYe3a/KZ166B5\n86pTs2b5leVbVyr2u2LWcJwIMmz9+vy/XCumL77Y+Ndsbttfflnsd2pDUsMnl80pa9YsiSlLU/Pm\n0KLFho/1lW3sa7KQ9DOTCO65B264oe46+ZxJW1+dhmijIeusW1f3L+dC2Gqr/Ke2bTcs23rrjWuj\n+tSsWbJtFdP69VXnayvbmLqFbPPLL2HNmo1rc/36mi6bbNgpq5o1a/jksqnLjzkGBg1q+G3MTCJo\n1y6/4Qbyyf711WmINhqqTrNmDfPlWtNUU5stWmTjF1RWFTrZbMy0fj2UlyfTunVVH+srK/TymspW\nr978dXbq5ESwWU44IZnMbNNVdMtYcRRqz8xnDZmZbSEKlYSdCMzMMs6JwMws45wIzMwyzonAzCzj\nnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wI\nzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws4wqaCCQdJWmOpHmSxtawfFdJz0l6TdIsSd8q\nZDxmZrahgiUCSc2Bm4GjgZ7AcEk9q1X7D2BSRAwATgPGFSoeMzOrWSH3CAYD8yJifkSsBSYCw6rV\nCWDb9Pl2wOICxmNmZjUoZCLoBLyfM78wLct1JXC6pIXAE8AFNTUkaZSkMkllS5cuLUSsZmaZVeyD\nxcOB30ZEZ+BbwO8kbRBTRIyPiNKIKO3YsWOjB2lm1pQVMhEsArrkzHdOy3KdDUwCiIi/A62ADgWM\nyczMqilkIpgG7CGpm6StSA4GT6lW5z3gMABJPUgSgft+zMwaUcESQUSUA+cDTwFvkZwd9KakqyQd\nm1b7d+BcSTOB+4GRERGFisnMzDbUopCNR8QTJAeBc8t+mvN8NrB/IWMwM7O6FftgsZmZFZkTgZlZ\nxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5\nEZiZZZwTgZlZxjkRmJllnBOBmVnGORGYmWWcE4GZWcY5EZiZZZwTgZlZxjkRmJllnBOBmVnGORGY\nmWVci3wrSuoEdM19TUS8UIigzMys8eSVCCT9F3AqMBtYlxYH4ERgZraFy3eP4Dige0R8UchgzMys\n8eV7jGA+0LKQgZiZWXHku0ewCpgh6Vmgcq8gIi4sSFRmZtZo8k0EU9LJzMyamLwSQUTcI2krYM+0\naE5EfFm4sMzMrLHkdYxA0sHAXOBmYBzwjqQD83jdUZLmSJonaWwtdU6RNFvSm5J+vxGxm5lZA8i3\na+i/gaERMQdA0p7A/cDetb1AUnOSxHEEsBCYJmlKRMzOqbMH8H+B/SPin5J22rTNMDOzTZXvWUMt\nK5IAQES8Q/1nEQ0G5kXE/IhYC0wEhlWrcy5wc0T8M233ozzjMTOzBpJvIiiTdIekg9PpdqCsntd0\nAt7PmV+YluXaE9hT0v9IelnSUTU1JGmUpDJJZUuXLs0zZDMzy0e+iWA0yVXFF6bT7LRsc7UA9gAO\nBoYDt0tqX71SRIyPiNKIKO3YsWMDrNbMzCrke9bQF8B16ZSvRUCXnPnOaVmuhcDU9Aykf0h6hyQx\nTNuI9ZiZ2Waoc49A0qT08XVJs6pP9bQ9DdhDUrf01NPT2PBahEdI9gaQ1IGkq2j+JmyHmZltovr2\nCC5KH7+zsQ1HRLmk84GngObAXRHxpqSrgLKImJIuGyqpYjC7H0fE8o1dl5mZbTpFRP2VpG2A1RGx\nPj11dC/gyWJcVFZaWhplZfUdpzYzs1ySpkdEaU3L8j1Y/ALQKr0nwdPAd4HfNkx4ZmZWTPkmAkXE\nKuAEYFxEnAz0KlxYZmbWWPJOBJL2A0YAj6dlzQsTkpmZNaZ8E8EYkqEgJqcHfHcDnitcWGZm1ljy\nvY7gr8Bfc+bnk1xYZmZmW7g6E4GkGyJijKQ/ktyjuIqIOLZgkZmZWaOob4/gd+njtYUOxMzMiqPO\nRBAR09OnZaTXEUDlENNbFzg2MzNrBPkeLH4WaJMz3xp4puHDMTOzxpZvImgVEZ9XzKTP29RR38zM\nthD5JoKVkgZWzEjaG1hdmJDMzKwx5XuryjHAHyQtBgTsDJxasKjMzKzR5HsdwTRJewHd06I5xRhw\nzszMGl5eXUOS2gCXABdFxBtAiaSNHprazMy+fvI9RnA3sBbYL51fBPy8IBGZmVmjyjcRfDMifgl8\nCZCORKqCRWVmZo0m30SwVlJr0mEmJH0T+KJgUZmZWaPJ96yhK4A/AV0kTQD2B0YWKigzM2s89SYC\nSQLeJrkpzb4kXUIXRcSyAsdmZmaNoN5EEBEh6YmI6MNXN6UxM7MmIt9jBK9KGlTQSMzMrCjyPUaw\nD3C6pAXASpLuoYiIvoUKzMzMGke+ieDIgkZhZmZFU98dyloB5wG7A68Dd0ZEeWMEZmZmjaO+YwT3\nAKUkSeBo4L8LHpGZmTWq+rqGeqZnCyHpTuCVwodkZmaNqb49gsoRRt0lZGbWNNW3R9BP0qfpcwGt\n0/mKs4a2LWh0ZmZWcPXdvL55YwViZmbFke8FZWZm1kQ5EZiZZVxBE4GkoyTNkTRP0tg66p0oKSSV\nFjIeMzPbUMESgaTmwM0k1x/0BIZL6llDvXbARcDUQsViZma1K+QewWBgXkTMj4i1wERgWA31/hP4\nL2BNAWMxM7NaFDIRdALez5lfmJZVkjQQ6BIRdQ5vLWmUpDJJZUuXLm34SM3MMqxoB4slNQOuA/69\nvroRMT4iSiOitGPHjoUPzswsQwqZCBYBXXLmO6dlFdoBvYHn0+Gt9wWm+ICxmVnjKmQimAbsIamb\npK2A04ApFQsj4pOI6BARJRFRArwMHBsRZQWMyczMqilYIkjHJjofeAp4C5gUEW9KukrSsYVar5mZ\nbZx8b0yzSSLiCeCJamU/raXuwYWMxczMauYri83MMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDM\nLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzj\nnAjMzDLOicDMLOOcCMzMMs6JwMws45wIzMwyzonAzCzjnAjMzDLOicDMLOOcCMzMMs6JwMws45wI\nzMwyzonAzCzjnAjMzDLOicDMLOMKmggkHSVpjqR5ksbWsPyHkmZLmiXpWUldCxmPmZltqGCJQFJz\n4GbgaKAnMFxSz2rVXgNKI6Iv8CDwy0LFY2ZmNSvkHsFgYF5EzI+ItcBEYFhuhYh4LiJWpbMvA50L\nGI+ZmdWgkImgE/B+zvzCtKw2ZwNP1rRA0ihJZZLKli5d2oAhmpnZ1+JgsaTTgVLgVzUtj4jxEVEa\nEaUdO3Zs3ODMzJq4FgVsexHQJWe+c1pWhaTDgcuAgyLii01Z0ZdffsnChQtZs2bNJgVqTVOrVq3o\n3LkzLVu2LHYoZl9rhUwE04A9JHUjSQCnAf+WW0HSAOA24KiI+GhTV7Rw4ULatWtHSUkJkjYnZmsi\nIoLly5ezcOFCunXrVuxwzL7WCtY1FBHlwPnAU8BbwKSIeFPSVZKOTav9CmgL/EHSDElTNmVda9as\nYccdd3QSsEqS2HHHHb2XaJaHQu4REBFPAE9UK/tpzvPDG2pdTgJWnT8TZvn5WhwsNjOz4slkIpgw\nAUpKoFmz5HHChM1rb/ny5fTv35/+/fuz884706lTp8r5tWvX5tXGmWeeyZw5c+qsc/PNNzNhc4NN\nDRkyhO7du9OvXz+GDBnC3LlzN7vNdevWccABB9RZ58gjj+Szzz7b7HWZWcNRRBQ7ho1SWloaZWVl\nVcreeustevTokdfrJ0yAUaNg1aqvytq0gfHjYcSIzY/vyiuvpG3btvzoRz+qUh4RRATNmn09cu+Q\nIUP4zW9+Q//+/Rk3bhzPPPMMDz/8cJU65eXltGhR0N7DgtuYz4ZZUyZpekSU1rTs6/Gt1Iguu6xq\nEoBk/rLLGn5d8+bNo2fPnowYMYJevXqxZMkSRo0aRWlpKb169eKqq66qrDtkyBBmzJhBeXk57du3\nZ+zYsfTr14/99tuPjz5KTqj6j//4D2644YbK+mPHjmXw4MF0796dv/3tbwCsXLmSE088kZ49e3LS\nSSdRWlrKjBkz6ozzwAMPZN68eQB07tyZsWPHMmDAACZPnszcuXM58sgj2XvvvTnwwAN55513APjg\ngw8YNmwYffv2pV+/fkydOrUydoBFixYxZMgQ+vfvT+/evSvj69y5MytWrADgl7/8Jb1796Z3797c\ndNNNle9Z7969Ofvss+nVqxdHH320D/iaFVjmEsF7721c+eZ6++23ufjii5k9ezadOnXiF7/4BWVl\nZcycOZM///nPzJ49e4PXfPLJJxx00EHMnDmT/fbbj7vuuqvGtiOCV155hV/96leVSeWmm25i5513\nZvbs2Vx++eW89tpr9cb4xz/+kT59+lTO77TTTrz22mucfPLJjBo1inHjxjF9+nSuueYazj//fAB+\n8IMfcMQRRzBr1iymT5++wa/u++67j2OOOYYZM2Ywc+ZM+vbtW2X51KlTmTBhAtOmTePvf/8748aN\n4/XXXwdgzpw5jBkzhjfffJPWrVvzyCOP1LsNZrbptuz9/k2w667w7rs1lxfCN7/5TUpLv9obu//+\n+7nzzjspLy9n8eLFzJ49m549q47F17p1a44++mgA9t57b1588cUa2z7hhBMq6yxYsACAl156iUsu\nuQSAfv360atXr1pjO/XUU2ndujW77bZb5S/yinKAFStW8PLLL3PiiSdWLisvLwfg+eefZ+LEiQC0\naNGCbbfdtnIZwKBBg/j+97/PmjVrOO644+jXr1+Vdb/00kuceOKJtG7dGoDjjjuOF198kaFDh7L7\n7rtXJqbcbTOzwshcIrj66pqPEVx9dWHWt80221Q+nzt3Lr/+9a955ZVXaN++PaeffnqN3R5bbbVV\n5fPmzZtX+YLNtfXWW9dbpy4PPPAA/fv3rzXmiKBDhw61di3VdXrmoYceyvPPP8/jjz/OGWecwU9+\n8hNG5HkQpmK7YNO3zczyl7muoREjkgPDXbuClDw21IHi+nz66ae0a9eObbfdliVLlvDUU081+Dr2\n339/Jk2aBMDrr79eY9dTvrbffnt22WUXJk+eDMD69euZOXMmAIcccgi33norkJwt9Omnn1Z57bvv\nvsvOO+/MqFGjOPPMMzfoojrggAOYPHkyq1ev5vPPP+fRRx+t94wjMyuMzCUCSL70FyyA9euTx8ZI\nAgADBw6kZ8+e7LXXXpxxxhnsv//+Db6OCy64gEWLFtGzZ09+9rOf0bNnT7bbbrtNbm/ixInceuut\nld1Mjz32GAC/+c1veOqpp+jTpw+lpaW8/fbbVV737LPP0q9fPwYMGMDDDz/MBRdcUGX54MGDGT58\nOIMGDWLfffdl9OjRVY5TmFnjydzpo01deXk55eXltGrVirlz5zJ06FDmzp27xZ8Guqn82TBL1HX6\naDa/HZqwzz//nMMOO4zy8nIigttuuy2zScDM8uNviCamffv2TJ8+vdhhmNkWJJPHCMzM7CtOBGZm\nGedEYGaWcU4EZmYZ50TQAA455JANLg674YYbGD16dJ2va9u2LQCLFy/mpJNOqrHOwQcfTPXTZau7\n4YYbWJVzqfS3vvWtyoHdNseVV15ZOaR27969mTJlk24gt4FzzjmnzgvdfvrTn/LMM880yLrMrH5O\nBA1g+PDhlePuVJg4cSLDhw/P6/Xf+MY3ePDBBzd5/dUTwRNPPFE5Cujmuvjii5kxYwZ/+MMfOOus\ns1i/fn2V5Zsy/MMdd9yxwfhKua666ioOP7zBbl5nZvVocolgzBg4+OCGncaMqXudJ510Eo8//njl\nTWgWLFjA4sWLOeCAAyrP6x84cCB9+vTh0Ucf3eD1CxYsoHfv3gCsXr2a0047jR49enD88cezevXq\nynqjR4+uHML6iiuuAODGG29k8eLFHHLIIRxyyCEAlJSUsGzZMgCuu+66yqGeK4awXrBgAT169ODc\nc8+lV69eDB06tMp6atKjRw9atGjBsmXLGDlyJOeddx777LMPP/nJT1i5ciVnnXUWgwcPZsCAAZXb\nuG7dOn70ox/Ru3dv+vbtWzmwXcVezrp16xg5ciS9e/emT58+XH/99QCMHDmyMjE+++yzDBgwgD59\n+nDWWWfxxRdfVG7jFVdcUfm+Vr+y2czy5+sIGsAOO+zA4MGDefLJJxk2bBgTJ07klFNOQRKtWrVi\n8uTJbLvttixbtox9992XY489ttYB22655RbatGnDW2+9xaxZsxg4cGDlsquvvpoddtiBdevWcdhh\nhzFr1iwuvPBCrrvuOp577jk6dOhQpa3p06dz9913M3XqVCKCffbZh4MOOojtt9+euXPncv/993P7\n7bdzyimn8NBDD3H66afXuo1Tp06lWbNmdOzYEYCFCxfyt7/9jebNm3PppZdy6KGHctddd7FixQoG\nDx7M4Ycfzr333suCBQuYMWMGLVq04OOPP67S5owZM1i0aBFvvPEGwAbdWWvWrGHkyJE8++yz7Lnn\nnpxxxhnccsstjEkzc4cOHXj11VcZN24c1157LXfccUeefzEzy9XkEkH6o7fRVXQPVSSCO++8E0hG\n8Lz00kt54YUXaNasGYsWLeLDDz9k5513rrGdF154gQsvvBCAvn37VhnHf9KkSYwfP57y8nKWLFnC\n7NmzNxjnP9dLL73E8ccfXzma6AknnMCLL77IscceS7du3SpHHq1rqOfrr7+e++67j3bt2vHAAw9U\nJrCTTz6Z5s2bA/D0008zZcoUrr32WiD5An/vvfd45plnOO+88yqvbN5hhx2qtL3bbrsxf/58Lrjg\nAr797W8zdOjQKsvnzJlDt27d2HPPPQH43ve+x80331yZCHKH4a5+dzUzy1+TSwTFMmzYMC6++GJe\nffVVVq1axd577w3AhAkTWLp0KdOnT6dly5aUlJRs0h23/vGPf3Dttdcybdo0tt9+e0aOHLlZd+6q\nPtRzbV1DF1988Qa33YSqw2tHBA899BDdu3ffqBi23357Zs6cyVNPPcWtt97KpEmTar0JT002dxhu\nM0s0uWMExdK2bVsOOeQQzjrrrCoHiT/55BN22mknWrZsyXPPPce7Nd0VJ8eBBx7I73//ewDeeOMN\nZs2aBSRDWG+zzTZst912fPjhhzz55JOVr2nXrl2NN4Q/4IADeOSRR1i1ahUrV65k8uTJBRnq+cgj\nj+Smm26iYgDDiiGnjzjiCG677bbKL+nqXUPLli1j/fr1nHjiifz85z/n1VdfrbK8e/fuLFiwoPI2\nmr/73e846KCDGjx+s6zzHkEDGj58OMcff3yVM4hGjBjBMcccUzlc81577VVnG6NHj+bMM8+kR48e\n9OjRo3LPomJI57322osuXbpUGcJ61KhRHHXUUXzjG9/gueeeqywfOHAgI0eOZPDgwUBy2uaAAQMa\n/I5fl19+OWPGjKFv376sX7+ebt268dhjj3HOOefwzjvv0LdvX1q2bMm5555beatLSO5rfOaZZ1ae\niXTNNddUabdVq1bcfffdnD5TFH0AAAYASURBVHzyyZSXlzNo0CDOO++8Bo3dzDwMtTVx/myYJeoa\nhtpdQ2ZmGedEYGaWcU0mEWxpXVxWeP5MmOWnSSSCVq1asXz5cv/jW6WIYPny5bRq1arYoZh97TWJ\ns4Y6d+7MwoULWbp0abFDsa+RVq1a0blz52KHYfa11yQSQcuWLenWrVuxwzAz2yIVtGtI0lGS5kia\nJ2lsDcu3lvRAunyqpJJCxmNmZhsqWCKQ1By4GTga6AkMl1R97OGzgX9GxO7A9cB/FSoeMzOrWSH3\nCAYD8yJifkSsBSYCw6rVGQbckz5/EDhMtQ3LaWZmBVHIYwSdgPdz5hcC+9RWJyLKJX0C7Agsy60k\naRQwKp39XNKcTYypQ/W2M8DbnA3e5mzYnG3uWtuCLeJgcUSMB8ZvbjuSymq7xLqp8jZng7c5Gwq1\nzYXsGloEdMmZ75yW1VhHUgtgO2B5AWMyM7NqCpkIpgF7SOomaSvgNKD63c+nAN9Ln58E/CV8VZiZ\nWaMqWNdQ2ud/PvAU0By4KyLelHQVUBYRU4A7gd9Jmgd8TJIsCmmzu5e2QN7mbPA2Z0NBtnmLG4ba\nzMwaVpMYa8jMzDadE4GZWcZlIhFIukvSR5LeKHYsjUVSF0nPSZot6U1JFxU7pkKT1ErSK5Jmptv8\ns2LH1BgkNZf0mqTHih1LY5C0QNLrkmZIKqv/FVs+Se0lPSjpbUlvSdqvQdvPwjECSQcCnwP3RkTv\nYsfTGCTtAuwSEa9KagdMB46LiNlFDq1g0qvSt4mIzyW1BF4CLoqIl4scWkFJ+iFQCmwbEd8pdjyF\nJmkBUBoRmbmYTNI9wIsRcUd6FmabiFjRUO1nYo8gIl4gOSspMyJiSUS8mj7/DHiL5EruJisSn6ez\nLdOpSf/SkdQZ+DZwR7FjscKQtB1wIMlZlkTE2oZMApCRRJB16aiuA4CpxY2k8NJukhnAR8CfI6Kp\nb/MNwE+A9cUOpBEF8LSk6enwM01dN2ApcHfaBXiHpG0acgVOBE2cpLbAQ8CYiPi02PEUWkSsi4j+\nJFeyD5bUZLsCJX0H+Cgiphc7lkY2JCIGkoxs/IO067cpawEMBG6JiAHASmCDYf03hxNBE5b2kz8E\nTIiIh4sdT2NKd52fA44qdiwFtD9wbNpnPhE4VNJ9xQ2p8CJiUfr4ETCZZKTjpmwhsDBn7/ZBksTQ\nYJwImqj0wOmdwFsRcV2x42kMkjpKap8+bw0cAbxd3KgKJyL+b0R0jogSkqvy/xIRpxc5rIKStE16\n8gNp98hQoEmfDRgRHwDvS+qeFh0GNOhJH1vE6KObS9L9wMFAB0kLgSsi4s7iRlVw+wPfBV5P+8wB\nLo2IJ4oYU6HtAtyT3hSpGTApIjJxSmWG/AswOb1tSQvg9xHxp+KG1CguACakZwzNB85syMYzcfqo\nmZnVzl1DZmYZ50RgZpZxTgRmZhnnRGBmlnFOBGZmGedEYJaStC4d0bJiarCrNyWVZGn0W9uyZOI6\nArM8rU6HpzDLFO8RmNUjHf/+l+kY+K9I2j0tL5H0F0mzJD0rade0/F8kTU7vizBT0r+mTTWXdHt6\nr4Sn06ufkXRhet+IWZImFmkzLcOcCMy+0rpa19CpOcs+iYg+wG9IRvwEuAm4JyL6AhOAG9PyG4G/\nRkQ/kjFh3kzL9wBujohewArgxLR8LDAgbee8Qm2cWW18ZbFZStLnEdG2hvIFwKERMT8dyO+DiNhR\n0jKSm/98mZYviYgOkpYCnSPii5w2SkiGxd4jnb8EaBkRP5f0J5IbJz0CPJJzTwWzRuE9ArP8RC3P\nN8YXOc/X8dUxum8DN5PsPUyT5GN31qicCMzyc2rO49/T538jGfUTYATwYvr8WWA0VN4oZ7vaGpXU\nDOgSEc8BlwDbARvslZgVkn95mH2ldc5IrQB/ioiKU0i3lzSL5Ff98LTsApK7Rv2Y5A5SFSNCXgSM\nl3Q2yS//0cCSWtbZHLgvTRYCbmzo2xCa1cfHCMzqkcWbpVu2uGvIzCzjvEdgZpZx3iMwM8s4JwIz\ns4xzIjAzyzgnAjOzjHMiMDPLuP8PLINHJSWFi1kAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i5Sn8FybpyqL",
        "colab_type": "text"
      },
      "source": [
        "## Parameter Optimisation\n",
        "\n",
        "You can of course use some of the previously used optimisation methods (e.g. Hyperopt), but here I'll introduce some new ones."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2iNQo8GLEt4E",
        "colab_type": "text"
      },
      "source": [
        "### Talos\n",
        "\n",
        "**TODO**\n",
        "- do an example with talos as seems very simple to add (https://github.com/autonomio/talos)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FfcvWl47bTrk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M1F2ofMLQ2Eh",
        "colab_type": "text"
      },
      "source": [
        "### HpBandSter\n",
        "\n",
        "First we'll set up the configuration space of the possible parameter values that could be selected."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6RwQWYQ1S1_e",
        "colab_type": "code",
        "outputId": "54a2390b-1615-4381-bfc4-ef96e01e92f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "source": [
        "import ConfigSpace as CS\n",
        "import ConfigSpace.hyperparameters as CSH\n",
        "\n",
        "def get_configspace():\n",
        "      \n",
        "    cs = CS.ConfigurationSpace()\n",
        "\n",
        "    # --------------\n",
        "    # General Params\n",
        "    # --------------\n",
        "    learning_rate = CSH.UniformFloatHyperparameter('learning_rate', lower=0.01, \n",
        "                                                   upper=0.2, default_value=0.1, \n",
        "                                                   log=True)\n",
        "    batch_size = CSH.CategoricalHyperparameter('batch_size', [32, 64])\n",
        "    cs.add_hyperparameters([learning_rate, batch_size])\n",
        "    \n",
        "    # ------------\n",
        "    # Dense Layers\n",
        "    # ------------\n",
        "    dense_layers = CSH.UniformIntegerHyperparameter('dense_layers', lower=1, \n",
        "                                                    upper=5, default_value=2)\n",
        "    units_first = CSH.UniformIntegerHyperparameter('units_first', lower=3, \n",
        "                                                   upper=300, default_value=300) # first bigger than hidden\n",
        "    units_hidden = CSH.UniformIntegerHyperparameter('units_hidden', lower=2, \n",
        "                                                    upper=300, default_value=100)\n",
        "\n",
        "    cs.add_hyperparameters([dense_layers, units_first, units_hidden])\n",
        "    \n",
        "    # make units_hidden an active hyperparameter if dense_layers is greater than 1\n",
        "    cs.add_conditions([CS.GreaterThanCondition(units_hidden, dense_layers, 1)])\n",
        "\n",
        "    return cs\n",
        "\n",
        "cs = get_configspace()\n",
        "config = cs.sample_configuration().get_dictionary()\n",
        "print(config)"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'batch_size': 64, 'dense_layers': 4, 'learning_rate': 0.01833276507683601, 'units_first': 46, 'units_hidden': 109}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zcoQhMdkVo4L",
        "colab_type": "code",
        "outputId": "5e744c68-d067-4ba4-c929-17a248bd6381",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 816
        }
      },
      "source": [
        "from tensorflow.keras.layers import Input\n",
        "\n",
        "def create_model(config, n_feat):\n",
        "  model = Sequential()\n",
        "\n",
        "  # -----------\n",
        "  # Input Layer\n",
        "  # -----------\n",
        "  model.add(Input(shape=n_feat))\n",
        "\n",
        "  # ------------\n",
        "  # Dense Layers\n",
        "  # ------------\n",
        "  for dense_layer in range(config['dense_layers']):\n",
        "    if dense_layer == 0:\n",
        "      units = config['units_first']\n",
        "    else:\n",
        "      units = config['units_hidden']\n",
        "    model.add(Dense(units=units))\n",
        "    \n",
        "    model.add(BatchNormalization())\n",
        "    model.add(Activation('elu'))\n",
        "    \n",
        "    model.add(Dropout(rate=0.5))\n",
        "\n",
        "  # ------------\n",
        "  # Output Layer\n",
        "  # ------------\n",
        "  model.add(Dense(units=1, activation = 'sigmoid'))\n",
        "  \n",
        "  return model\n",
        "\n",
        "run_id = 'test'\n",
        "test_model = create_model(config, X_train.shape[1])\n",
        "test_model.summary()"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_4 (Dense)              (None, 46)                59754     \n",
            "_________________________________________________________________\n",
            "batch_normalization_4 (Batch (None, 46)                184       \n",
            "_________________________________________________________________\n",
            "activation_4 (Activation)    (None, 46)                0         \n",
            "_________________________________________________________________\n",
            "dropout_3 (Dropout)          (None, 46)                0         \n",
            "_________________________________________________________________\n",
            "dense_5 (Dense)              (None, 109)               5123      \n",
            "_________________________________________________________________\n",
            "batch_normalization_5 (Batch (None, 109)               436       \n",
            "_________________________________________________________________\n",
            "activation_5 (Activation)    (None, 109)               0         \n",
            "_________________________________________________________________\n",
            "dropout_4 (Dropout)          (None, 109)               0         \n",
            "_________________________________________________________________\n",
            "dense_6 (Dense)              (None, 109)               11990     \n",
            "_________________________________________________________________\n",
            "batch_normalization_6 (Batch (None, 109)               436       \n",
            "_________________________________________________________________\n",
            "activation_6 (Activation)    (None, 109)               0         \n",
            "_________________________________________________________________\n",
            "dropout_5 (Dropout)          (None, 109)               0         \n",
            "_________________________________________________________________\n",
            "dense_7 (Dense)              (None, 109)               11990     \n",
            "_________________________________________________________________\n",
            "batch_normalization_7 (Batch (None, 109)               436       \n",
            "_________________________________________________________________\n",
            "activation_7 (Activation)    (None, 109)               0         \n",
            "_________________________________________________________________\n",
            "dropout_6 (Dropout)          (None, 109)               0         \n",
            "_________________________________________________________________\n",
            "dense_8 (Dense)              (None, 1)                 110       \n",
            "=================================================================\n",
            "Total params: 90,459\n",
            "Trainable params: 89,713\n",
            "Non-trainable params: 746\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PHq7X9YaWvkV",
        "colab_type": "code",
        "outputId": "e6d77b5a-5a18-4abc-a337-764603c75c4e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        }
      },
      "source": [
        "from hpbandster.core.worker import Worker\n",
        "\n",
        "class MyWorker(Worker):\n",
        "  def __init__(self, **kwargs):\n",
        "    super().__init__(**kwargs)\n",
        "    \n",
        "    self.x_train = X_train\n",
        "    self.x_val = X_val\n",
        "    self.y_train = y_train\n",
        "    self.y_val = y_val\n",
        "\n",
        "    self.n_feat = self.x_train.shape[1]\n",
        "\n",
        "    setattr(self.__class__, 'get_configspace', staticmethod(get_configspace))\n",
        "  \n",
        "  \n",
        "  def compute(self, config, budget, working_directory, *args, **kwargs):\n",
        "    clear_session()\n",
        "\n",
        "    training_generator, steps_per_epoch =\\\n",
        "    balanced_batch_generator(self.x_train, self.y_train,\n",
        "                             batch_size=config['batch_size'])\n",
        "    \n",
        "    # Callbacks\n",
        "    early_stopping = EarlyStopping(monitor='val_loss',\n",
        "                                   patience=5, \n",
        "                                   verbose=1)\n",
        "\n",
        "    # Create Model\n",
        "    model = create_model(config, self.x_train.shape[1])\n",
        "\n",
        "    model.compile(optimizer=Adam(learning_rate=config['learning_rate']),\n",
        "                  metrics=['accuracy', 'Recall', 'Precision'],\n",
        "                  loss='binary_crossentropy')\n",
        "\n",
        "    # Train\n",
        "    history = model.fit_generator(generator=training_generator,\n",
        "                                  epochs=int(budget),\n",
        "                                  steps_per_epoch=steps_per_epoch,\n",
        "                                  validation_data = (self.x_val, self.y_val),\n",
        "                                  callbacks=[early_stopping],\n",
        "                                  verbose=1)\n",
        "  \n",
        "    # need to make the types of data floats not numpy.floats\n",
        "    history_dict = history.history\n",
        "    info_dict = {}\n",
        "    for key in history_dict.keys():\n",
        "        history_dict[key] = list(map(float, history_dict[key])) \n",
        "    info_dict.update(history_dict)\n",
        "\n",
        "    return ({'loss': 1- history_dict['val_Precision'][-1], # remember: HpBandSter always minimizes!\n",
        "             'info': info_dict})\n",
        "\n",
        "\n",
        "worker = MyWorker(run_id=run_id)\n",
        "res = worker.compute(config=config, budget=1, working_directory='.')\n",
        "print(res)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:09 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 15 steps, validate on 2411 samples\n",
            "11/15 [=====================>........] - ETA: 0s - loss: 0.2946 - accuracy: 0.8764 - Recall: 0.8494 - Precision: 0.8979"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r15/15 [==============================] - 2s 159ms/step - loss: 0.2519 - accuracy: 0.9042 - Recall: 0.8810 - Precision: 0.9234 - val_loss: 6.9351 - val_accuracy: 0.8105 - val_Recall: 1.0000 - val_Precision: 0.3055\n",
            "{'loss': 0.6945288777351379, 'info': {'loss': [0.25186533890664575], 'accuracy': [0.9041666388511658], 'Recall': [0.8810020685195923], 'Precision': [0.9234135746955872], 'val_loss': [6.9351040721709225], 'val_accuracy': [0.8104521036148071], 'val_Recall': [1.0], 'val_Precision': [0.30547112226486206]}}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YFnvzmFMR1OD",
        "colab_type": "code",
        "outputId": "1b7acc82-9261-4208-a3af-22392242b05f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "from hpbandster.optimizers.bohb import BOHB\n",
        "import hpbandster.core.nameserver as hpns\n",
        "import hpbandster.core.result as hpres\n",
        "import pickle\n",
        "\n",
        "host = '127.0.0.1'\n",
        "port = 9090\n",
        "save_dir = '.'\n",
        "run_id = 'MLP'\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "  result_logger = hpres.json_result_logger(directory=save_dir, overwrite=True)\n",
        "  \n",
        "  NS = hpns.NameServer(run_id=run_id, \n",
        "                       working_directory = save_dir,\n",
        "                       host=host,\n",
        "                       port=port)\n",
        "  NS.start()\n",
        "  \n",
        "  worker = MyWorker(run_id=run_id, nameserver =host)\n",
        "  worker.run(background=True)\n",
        "\n",
        "  bohb = BOHB(configspace = worker.get_configspace(),\n",
        "              run_id = run_id, nameserver=host,\n",
        "              min_budget=1, max_budget=5, # budget in this case means epochs!\n",
        "              result_logger=result_logger,\n",
        "             )\n",
        "  res = bohb.run(n_iterations=5)\n",
        "  \n",
        "  bohb.shutdown(shutdown_workers=True)\n",
        "  NS.shutdown()"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "19:27:22 DISPATCHER: started the 'discover_worker' thread\n",
            "19:27:22 wait_for_workers trying to get the condition\n",
            "19:27:22 WORKER: Connected to nameserver <Pyro4.core.Proxy at 0x7f3fb0dadeb8; connected IPv4; for PYRO:Pyro.NameServer@127.0.0.1:9090>\n",
            "19:27:22 DISPATCHER: started the 'job_runner' thread\n",
            "19:27:22 WORKER: No dispatcher found. Waiting for one to initiate contact.\n",
            "19:27:22 WORKER: start listening for jobs\n",
            "19:27:22 DISPATCHER: Pyro daemon running on localhost:40829\n",
            "19:27:22 DISPATCHER: Starting worker discovery\n",
            "19:27:22 DISPATCHER: Found 1 potential workers, 0 currently in the pool.\n",
            "19:27:22 DISPATCHER: discovered new worker, hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:22 HBMASTER: number of workers changed to 1\n",
            "19:27:22 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:22 Enough workers to start this run!\n",
            "19:27:22 adjust_queue_size: lock accquired\n",
            "19:27:22 HBMASTER: starting run at 1583609242.8097627\n",
            "19:27:22 HBMASTER: adjusted queue size to (0, 1)\n",
            "19:27:22 DISPATCHER: Finished worker discovery\n",
            "19:27:22 start sampling a new configuration.\n",
            "19:27:22 DISPATCHER: Trying to submit another job.\n",
            "19:27:22 done sampling a new configuration.\n",
            "19:27:22 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:22 HBMASTER: schedule new run for iteration 0\n",
            "19:27:22 HBMASTER: trying submitting job (0, 0, 0) to dispatcher\n",
            "19:27:22 HBMASTER: submitting job (0, 0, 0) to dispatcher\n",
            "19:27:22 DISPATCHER: trying to submit job (0, 0, 0)\n",
            "19:27:22 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:22 HBMASTER: job (0, 0, 0) submitted to dispatcher\n",
            "19:27:22 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:22 DISPATCHER: Trying to submit another job.\n",
            "19:27:22 DISPATCHER: starting job (0, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:22 DISPATCHER: job (0, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:22 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:22 WORKER: start processing job (0, 0, 0)\n",
            "19:27:22 WORKER: args: ()\n",
            "19:27:22 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.013676652928876662, 'units_first': 166, 'units_hidden': 220}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:23 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "18/30 [=================>............] - ETA: 1s - loss: 0.2799 - accuracy: 0.9184 - Recall: 0.8834 - Precision: 0.9470"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 105ms/step - loss: 0.2039 - accuracy: 0.9396 - Recall: 0.9158 - Precision: 0.9603 - val_loss: 3.7214 - val_accuracy: 0.9249 - val_Recall: 0.9851 - val_Precision: 0.5266\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:26 WORKER: done with job (0, 0, 0), trying to register it.\n",
            "19:27:26 DISPATCHER: job (0, 0, 0) finished\n",
            "19:27:26 WORKER: registered result for job (0, 0, 0) with dispatcher\n",
            "19:27:26 DISPATCHER: register_result: lock acquired\n",
            "19:27:26 DISPATCHER: job (0, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:26 job_id: (0, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.013676652928876662, 'units_first': 166, 'units_hidden': 220}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.4734042286872864, 'info': {'loss': [0.20386494317402443], 'accuracy': [0.9395833611488342], 'Recall': [0.9157894849777222], 'Precision': [0.9602649211883545], 'val_loss': [3.7213830690707144], 'val_accuracy': [0.9249274134635925], 'val_Recall': [0.9850746393203735], 'val_Precision': [0.5265957713127136]}}\n",
            "exception: None\n",
            "\n",
            "19:27:26 job_callback for (0, 0, 0) started\n",
            "19:27:26 job_callback for (0, 0, 0) got condition\n",
            "19:27:26 DISPATCHER: Trying to submit another job.\n",
            "19:27:26 Only 1 run(s) for budget 1.666667 available, need more than 7 -> can't build model!\n",
            "19:27:26 HBMASTER: Trying to run another job!\n",
            "19:27:26 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:26 job_callback for (0, 0, 0) finished\n",
            "19:27:26 start sampling a new configuration.\n",
            "19:27:26 done sampling a new configuration.\n",
            "19:27:26 HBMASTER: schedule new run for iteration 0\n",
            "19:27:26 HBMASTER: trying submitting job (0, 0, 1) to dispatcher\n",
            "19:27:26 HBMASTER: submitting job (0, 0, 1) to dispatcher\n",
            "19:27:26 DISPATCHER: trying to submit job (0, 0, 1)\n",
            "19:27:26 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:26 HBMASTER: job (0, 0, 1) submitted to dispatcher\n",
            "19:27:26 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:26 DISPATCHER: Trying to submit another job.\n",
            "19:27:26 DISPATCHER: starting job (0, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:26 DISPATCHER: job (0, 0, 1) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:26 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:26 WORKER: start processing job (0, 0, 1)\n",
            "19:27:26 WORKER: args: ()\n",
            "19:27:26 WORKER: kwargs: {'config': {'batch_size': 64, 'dense_layers': 3, 'learning_rate': 0.0260834334197807, 'units_first': 154, 'units_hidden': 27}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:27 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 15 steps, validate on 2411 samples\n",
            " 8/15 [===============>..............] - ETA: 1s - loss: 0.2748 - accuracy: 0.8809 - Recall: 0.8554 - Precision: 0.8950 "
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r15/15 [==============================] - 2s 134ms/step - loss: 0.2095 - accuracy: 0.9187 - Recall: 0.8958 - Precision: 0.9389 - val_loss: 1.9503 - val_accuracy: 0.8706 - val_Recall: 0.9900 - val_Precision: 0.3910\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:29 WORKER: done with job (0, 0, 1), trying to register it.\n",
            "19:27:29 WORKER: registered result for job (0, 0, 1) with dispatcher\n",
            "19:27:29 DISPATCHER: job (0, 0, 1) finished\n",
            "19:27:29 DISPATCHER: register_result: lock acquired\n",
            "19:27:29 DISPATCHER: job (0, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:29 job_id: (0, 0, 1)\n",
            "kwargs: {'config': {'batch_size': 64, 'dense_layers': 3, 'learning_rate': 0.0260834334197807, 'units_first': 154, 'units_hidden': 27}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.6090373396873474, 'info': {'loss': [0.20954266314705214], 'accuracy': [0.918749988079071], 'Recall': [0.8958333134651184], 'Precision': [0.9388646483421326], 'val_loss': [1.950342891012478], 'val_accuracy': [0.8705931305885315], 'val_Recall': [0.9900497794151306], 'val_Precision': [0.3909626603126526]}}\n",
            "exception: None\n",
            "\n",
            "19:27:29 job_callback for (0, 0, 1) started\n",
            "19:27:29 DISPATCHER: Trying to submit another job.\n",
            "19:27:29 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:29 job_callback for (0, 0, 1) got condition\n",
            "19:27:29 Only 2 run(s) for budget 1.666667 available, need more than 7 -> can't build model!\n",
            "19:27:29 HBMASTER: Trying to run another job!\n",
            "19:27:29 job_callback for (0, 0, 1) finished\n",
            "19:27:29 start sampling a new configuration.\n",
            "19:27:29 done sampling a new configuration.\n",
            "19:27:29 HBMASTER: schedule new run for iteration 0\n",
            "19:27:29 HBMASTER: trying submitting job (0, 0, 2) to dispatcher\n",
            "19:27:29 HBMASTER: submitting job (0, 0, 2) to dispatcher\n",
            "19:27:29 DISPATCHER: trying to submit job (0, 0, 2)\n",
            "19:27:29 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:29 HBMASTER: job (0, 0, 2) submitted to dispatcher\n",
            "19:27:29 DISPATCHER: Trying to submit another job.\n",
            "19:27:29 DISPATCHER: starting job (0, 0, 2) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:29 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:29 DISPATCHER: job (0, 0, 2) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:29 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:29 WORKER: start processing job (0, 0, 2)\n",
            "19:27:29 WORKER: args: ()\n",
            "19:27:29 WORKER: kwargs: {'config': {'batch_size': 64, 'dense_layers': 3, 'learning_rate': 0.07359083315431945, 'units_first': 112, 'units_hidden': 220}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:29 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 15 steps, validate on 2411 samples\n",
            "14/15 [===========================>..] - ETA: 0s - loss: 0.3841 - accuracy: 0.9040 - Recall: 0.8843 - Precision: 0.9247"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r15/15 [==============================] - 2s 130ms/step - loss: 0.3600 - accuracy: 0.9104 - Recall: 0.8900 - Precision: 0.9286 - val_loss: 13.8978 - val_accuracy: 0.8577 - val_Recall: 0.9851 - val_Precision: 0.3680\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:31 WORKER: done with job (0, 0, 2), trying to register it.\n",
            "19:27:31 WORKER: registered result for job (0, 0, 2) with dispatcher\n",
            "19:27:31 DISPATCHER: job (0, 0, 2) finished\n",
            "19:27:31 DISPATCHER: register_result: lock acquired\n",
            "19:27:31 DISPATCHER: job (0, 0, 2) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:31 job_id: (0, 0, 2)\n",
            "kwargs: {'config': {'batch_size': 64, 'dense_layers': 3, 'learning_rate': 0.07359083315431945, 'units_first': 112, 'units_hidden': 220}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.6319702565670013, 'info': {'loss': [0.35997758073111374], 'accuracy': [0.9104166626930237], 'Recall': [0.8900414705276489], 'Precision': [0.9285714030265808], 'val_loss': [13.897789529684538], 'val_accuracy': [0.8577353954315186], 'val_Recall': [0.9850746393203735], 'val_Precision': [0.36802974343299866]}}\n",
            "exception: None\n",
            "\n",
            "19:27:31 job_callback for (0, 0, 2) started\n",
            "19:27:31 DISPATCHER: Trying to submit another job.\n",
            "19:27:31 job_callback for (0, 0, 2) got condition\n",
            "19:27:31 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:31 Only 3 run(s) for budget 1.666667 available, need more than 7 -> can't build model!\n",
            "19:27:31 HBMASTER: Trying to run another job!\n",
            "19:27:31 job_callback for (0, 0, 2) finished\n",
            "19:27:31 ITERATION: Advancing config (0, 0, 0) to next budget 5.000000\n",
            "19:27:31 HBMASTER: schedule new run for iteration 0\n",
            "19:27:31 HBMASTER: trying submitting job (0, 0, 0) to dispatcher\n",
            "19:27:31 HBMASTER: submitting job (0, 0, 0) to dispatcher\n",
            "19:27:31 DISPATCHER: trying to submit job (0, 0, 0)\n",
            "19:27:31 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:31 HBMASTER: job (0, 0, 0) submitted to dispatcher\n",
            "19:27:31 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:31 DISPATCHER: Trying to submit another job.\n",
            "19:27:31 DISPATCHER: starting job (0, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:31 DISPATCHER: job (0, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:31 WORKER: start processing job (0, 0, 0)\n",
            "19:27:31 WORKER: args: ()\n",
            "19:27:31 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:31 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.013676652928876662, 'units_first': 166, 'units_hidden': 220}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:32 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "Epoch 1/5\n",
            "12/30 [===========>..................] - ETA: 2s - loss: 0.2946 - accuracy: 0.9115 - Recall: 0.8980 - Precision: 0.9263"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 96ms/step - loss: 0.2271 - accuracy: 0.9375 - Recall: 0.9312 - Precision: 0.9430 - val_loss: 1.5710 - val_accuracy: 0.9332 - val_Recall: 0.9851 - val_Precision: 0.5562\n",
            "Epoch 2/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.1107 - accuracy: 0.9739 - Recall: 0.9749 - Precision: 0.9729 - val_loss: 0.2441 - val_accuracy: 0.9718 - val_Recall: 0.9851 - val_Precision: 0.7529\n",
            "Epoch 3/5\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.0785 - accuracy: 0.9781 - Recall: 0.9791 - Precision: 0.9770 - val_loss: 0.1312 - val_accuracy: 0.9813 - val_Recall: 0.9801 - val_Precision: 0.8277\n",
            "Epoch 4/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.0837 - accuracy: 0.9823 - Recall: 0.9875 - Precision: 0.9774 - val_loss: 0.0977 - val_accuracy: 0.9822 - val_Recall: 0.9602 - val_Precision: 0.8465\n",
            "Epoch 5/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.0695 - accuracy: 0.9823 - Recall: 0.9853 - Precision: 0.9791 - val_loss: 0.0827 - val_accuracy: 0.9834 - val_Recall: 0.9453 - val_Precision: 0.8676\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:38 WORKER: done with job (0, 0, 0), trying to register it.\n",
            "19:27:38 WORKER: registered result for job (0, 0, 0) with dispatcher\n",
            "19:27:38 DISPATCHER: job (0, 0, 0) finished\n",
            "19:27:38 DISPATCHER: register_result: lock acquired\n",
            "19:27:38 DISPATCHER: job (0, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:38 job_id: (0, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.013676652928876662, 'units_first': 166, 'units_hidden': 220}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.13242006301879883, 'info': {'loss': [0.22714193847884115, 0.11038919990893943, 0.07835960702383443, 0.08358196866754948, 0.06930161298627893], 'accuracy': [0.9375, 0.9739039540290833, 0.9780793190002441, 0.982254683971405, 0.982254683971405], 'Recall': [0.9312499761581421, 0.9749478101730347, 0.9790794849395752, 0.987525999546051, 0.9852941036224365], 'Precision': [0.9430379867553711, 0.9729166626930237, 0.9770354628562927, 0.9773662686347961, 0.9791231751441956], 'val_loss': [1.5710377627089884, 0.24414912849469328, 0.13121936356277722, 0.09772902116376123, 0.0827457167243654], 'val_accuracy': [0.9332227110862732, 0.971795916557312, 0.9813355207443237, 0.9821650981903076, 0.9834093451499939], 'val_Recall': [0.9850746393203735, 0.9850746393203735, 0.9800994992256165, 0.9601989984512329, 0.9452736377716064], 'val_Precision': [0.5561797618865967, 0.7528517246246338, 0.8277310729026794, 0.8464912176132202, 0.8675799369812012]}}\n",
            "exception: None\n",
            "\n",
            "19:27:38 job_callback for (0, 0, 0) started\n",
            "19:27:38 DISPATCHER: Trying to submit another job.\n",
            "19:27:38 job_callback for (0, 0, 0) got condition\n",
            "19:27:38 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:38 Only 1 run(s) for budget 5.000000 available, need more than 7 -> can't build model!\n",
            "19:27:38 HBMASTER: Trying to run another job!\n",
            "19:27:38 job_callback for (0, 0, 0) finished\n",
            "19:27:38 start sampling a new configuration.\n",
            "19:27:38 done sampling a new configuration.\n",
            "19:27:38 HBMASTER: schedule new run for iteration 1\n",
            "19:27:38 HBMASTER: trying submitting job (1, 0, 0) to dispatcher\n",
            "19:27:38 HBMASTER: submitting job (1, 0, 0) to dispatcher\n",
            "19:27:38 DISPATCHER: trying to submit job (1, 0, 0)\n",
            "19:27:38 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:38 HBMASTER: job (1, 0, 0) submitted to dispatcher\n",
            "19:27:38 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:38 DISPATCHER: Trying to submit another job.\n",
            "19:27:38 DISPATCHER: starting job (1, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:38 DISPATCHER: job (1, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:38 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:38 WORKER: start processing job (1, 0, 0)\n",
            "19:27:38 WORKER: args: ()\n",
            "19:27:38 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 1, 'learning_rate': 0.09251226741882937, 'units_first': 176}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:38 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "Epoch 1/5\n",
            "20/30 [===================>..........] - ETA: 0s - loss: 0.2649 - accuracy: 0.9500 - Recall: 0.9429 - Precision: 0.9550"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 2s 55ms/step - loss: 0.3185 - accuracy: 0.9427 - Recall: 0.9374 - Precision: 0.9473 - val_loss: 0.6500 - val_accuracy: 0.9361 - val_Recall: 0.9851 - val_Precision: 0.5673\n",
            "Epoch 2/5\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.2260 - accuracy: 0.9603 - Recall: 0.9686 - Precision: 0.9526 - val_loss: 0.4075 - val_accuracy: 0.9602 - val_Recall: 0.9801 - val_Precision: 0.6817\n",
            "Epoch 3/5\n",
            "30/30 [==============================] - 1s 17ms/step - loss: 0.1886 - accuracy: 0.9656 - Recall: 0.9581 - Precision: 0.9723 - val_loss: 0.5313 - val_accuracy: 0.9589 - val_Recall: 0.9801 - val_Precision: 0.6747\n",
            "Epoch 4/5\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.0857 - accuracy: 0.9729 - Recall: 0.9671 - Precision: 0.9792 - val_loss: 0.2417 - val_accuracy: 0.9826 - val_Recall: 0.9751 - val_Precision: 0.8412\n",
            "Epoch 5/5\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.0611 - accuracy: 0.9802 - Recall: 0.9812 - Precision: 0.9792 - val_loss: 0.3925 - val_accuracy: 0.9689 - val_Recall: 0.9801 - val_Precision: 0.7351\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:42 WORKER: done with job (1, 0, 0), trying to register it.\n",
            "19:27:42 WORKER: registered result for job (1, 0, 0) with dispatcher\n",
            "19:27:42 DISPATCHER: job (1, 0, 0) finished\n",
            "19:27:42 DISPATCHER: register_result: lock acquired\n",
            "19:27:42 DISPATCHER: job (1, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:42 job_id: (1, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 1, 'learning_rate': 0.09251226741882937, 'units_first': 176}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.26492536067962646, 'info': {'loss': [0.31846407874967325, 0.22567194427217047, 0.18859648323126724, 0.08589463430282558, 0.06096853832298664], 'accuracy': [0.9427083134651184, 0.9603340029716492, 0.9655532240867615, 0.9728600978851318, 0.980167031288147], 'Recall': [0.9373695254325867, 0.9685534834861755, 0.9580712914466858, 0.9671458005905151, 0.9812108278274536], 'Precision': [0.947257399559021, 0.9525772929191589, 0.9723404049873352, 0.9792099595069885, 0.9791666865348816], 'val_loss': [0.6499704485758052, 0.4075208088929279, 0.5312762814168759, 0.2416745709951062, 0.392502992280357], 'val_accuracy': [0.9361261129379272, 0.9601824879646301, 0.9589381814002991, 0.9825798273086548, 0.9688925743103027], 'val_Recall': [0.9850746393203735, 0.9800994992256165, 0.9800994992256165, 0.9751243591308594, 0.9800994992256165], 'val_Precision': [0.5673352479934692, 0.6816608905792236, 0.6746575236320496, 0.8412017226219177, 0.7350746393203735]}}\n",
            "exception: None\n",
            "\n",
            "19:27:42 job_callback for (1, 0, 0) started\n",
            "19:27:42 DISPATCHER: Trying to submit another job.\n",
            "19:27:42 job_callback for (1, 0, 0) got condition\n",
            "19:27:42 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:42 Only 2 run(s) for budget 5.000000 available, need more than 7 -> can't build model!\n",
            "19:27:42 HBMASTER: Trying to run another job!\n",
            "19:27:42 job_callback for (1, 0, 0) finished\n",
            "19:27:42 start sampling a new configuration.\n",
            "19:27:42 done sampling a new configuration.\n",
            "19:27:42 HBMASTER: schedule new run for iteration 1\n",
            "19:27:42 HBMASTER: trying submitting job (1, 0, 1) to dispatcher\n",
            "19:27:42 HBMASTER: submitting job (1, 0, 1) to dispatcher\n",
            "19:27:42 DISPATCHER: trying to submit job (1, 0, 1)\n",
            "19:27:42 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:42 HBMASTER: job (1, 0, 1) submitted to dispatcher\n",
            "19:27:42 DISPATCHER: Trying to submit another job.\n",
            "19:27:42 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:42 DISPATCHER: starting job (1, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:42 DISPATCHER: job (1, 0, 1) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:42 WORKER: start processing job (1, 0, 1)\n",
            "19:27:42 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:42 WORKER: args: ()\n",
            "19:27:42 WORKER: kwargs: {'config': {'batch_size': 64, 'dense_layers': 1, 'learning_rate': 0.07298463002156952, 'units_first': 228}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:42 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 15 steps, validate on 2411 samples\n",
            "Epoch 1/5\n",
            " 8/15 [===============>..............] - ETA: 0s - loss: 0.2263 - accuracy: 0.9062 - Recall: 0.8884 - Precision: 0.9177 "
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "15/15 [==============================] - 1s 99ms/step - loss: 0.2227 - accuracy: 0.9344 - Recall: 0.9202 - Precision: 0.9460 - val_loss: 3.7662 - val_accuracy: 0.7545 - val_Recall: 1.0000 - val_Precision: 0.2535\n",
            "Epoch 2/5\n",
            "15/15 [==============================] - 0s 18ms/step - loss: 0.0766 - accuracy: 0.9806 - Recall: 0.9761 - Precision: 0.9846 - val_loss: 1.0483 - val_accuracy: 0.8693 - val_Recall: 0.9900 - val_Precision: 0.3887\n",
            "Epoch 3/5\n",
            "15/15 [==============================] - 0s 17ms/step - loss: 0.0656 - accuracy: 0.9784 - Recall: 0.9785 - Precision: 0.9785 - val_loss: 1.1403 - val_accuracy: 0.8664 - val_Recall: 0.9950 - val_Precision: 0.3839\n",
            "Epoch 4/5\n",
            "15/15 [==============================] - 0s 17ms/step - loss: 0.0697 - accuracy: 0.9860 - Recall: 0.9871 - Precision: 0.9850 - val_loss: 0.6296 - val_accuracy: 0.9303 - val_Recall: 0.9851 - val_Precision: 0.5455\n",
            "Epoch 5/5\n",
            "15/15 [==============================] - 0s 18ms/step - loss: 0.0464 - accuracy: 0.9838 - Recall: 0.9826 - Precision: 0.9847 - val_loss: 0.5139 - val_accuracy: 0.9419 - val_Recall: 0.9900 - val_Precision: 0.5905\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:45 WORKER: done with job (1, 0, 1), trying to register it.\n",
            "19:27:45 WORKER: registered result for job (1, 0, 1) with dispatcher\n",
            "19:27:45 DISPATCHER: job (1, 0, 1) finished\n",
            "19:27:45 DISPATCHER: register_result: lock acquired\n",
            "19:27:45 DISPATCHER: job (1, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:45 job_id: (1, 0, 1)\n",
            "kwargs: {'config': {'batch_size': 64, 'dense_layers': 1, 'learning_rate': 0.07298463002156952, 'units_first': 228}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.4094955325126648, 'info': {'loss': [0.22269041563073794, 0.0661756451971845, 0.06465452882264124, 0.059760556638755756, 0.0480042051244326], 'accuracy': [0.934374988079071, 0.9805615544319153, 0.9784017205238342, 0.9859611392021179, 0.9838013052940369], 'Recall': [0.9201680421829224, 0.9760869741439819, 0.9784946441650391, 0.9870967864990234, 0.9825708270072937], 'Precision': [0.946004331111908, 0.984649121761322, 0.9784946441650391, 0.9849785566329956, 0.9847161769866943], 'val_loss': [3.76617438641884, 1.0483391805710707, 1.1403466147945809, 0.6295934391282545, 0.513859848620479], 'val_accuracy': [0.7544587254524231, 0.8693488240242004, 0.8664454817771912, 0.9303193688392639, 0.9419327974319458], 'val_Recall': [1.0, 0.9900497794151306, 0.9950248599052429, 0.9850746393203735, 0.9900497794151306], 'val_Precision': [0.253467857837677, 0.388671875, 0.383877158164978, 0.5454545617103577, 0.5905044674873352]}}\n",
            "exception: None\n",
            "\n",
            "19:27:45 job_callback for (1, 0, 1) started\n",
            "19:27:45 DISPATCHER: Trying to submit another job.\n",
            "19:27:45 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:45 job_callback for (1, 0, 1) got condition\n",
            "19:27:45 Only 3 run(s) for budget 5.000000 available, need more than 7 -> can't build model!\n",
            "19:27:45 HBMASTER: Trying to run another job!\n",
            "19:27:45 job_callback for (1, 0, 1) finished\n",
            "19:27:45 start sampling a new configuration.\n",
            "19:27:45 done sampling a new configuration.\n",
            "19:27:45 HBMASTER: schedule new run for iteration 2\n",
            "19:27:45 HBMASTER: trying submitting job (2, 0, 0) to dispatcher\n",
            "19:27:45 HBMASTER: submitting job (2, 0, 0) to dispatcher\n",
            "19:27:45 DISPATCHER: trying to submit job (2, 0, 0)\n",
            "19:27:45 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:45 HBMASTER: job (2, 0, 0) submitted to dispatcher\n",
            "19:27:45 DISPATCHER: Trying to submit another job.\n",
            "19:27:45 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:45 DISPATCHER: starting job (2, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:45 DISPATCHER: job (2, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:45 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:45 WORKER: start processing job (2, 0, 0)\n",
            "19:27:45 WORKER: args: ()\n",
            "19:27:45 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 4, 'learning_rate': 0.09400132689549366, 'units_first': 227, 'units_hidden': 65}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:46 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "20/30 [===================>..........] - ETA: 0s - loss: 0.3547 - accuracy: 0.8953 - Recall: 0.8857 - Precision: 0.9000"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 86ms/step - loss: 0.2989 - accuracy: 0.9125 - Recall: 0.9108 - Precision: 0.9146 - val_loss: 1.5125 - val_accuracy: 0.9303 - val_Recall: 0.9851 - val_Precision: 0.5455\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:48 WORKER: done with job (2, 0, 0), trying to register it.\n",
            "19:27:48 WORKER: registered result for job (2, 0, 0) with dispatcher\n",
            "19:27:48 DISPATCHER: job (2, 0, 0) finished\n",
            "19:27:48 DISPATCHER: register_result: lock acquired\n",
            "19:27:48 DISPATCHER: job (2, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:48 job_id: (2, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 4, 'learning_rate': 0.09400132689549366, 'units_first': 227, 'units_hidden': 65}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.45454543828964233, 'info': {'loss': [0.2989098639537891], 'accuracy': [0.9125000238418579], 'Recall': [0.910788357257843], 'Precision': [0.9145833253860474], 'val_loss': [1.5124968023045744], 'val_accuracy': [0.9303193688392639], 'val_Recall': [0.9850746393203735], 'val_Precision': [0.5454545617103577]}}\n",
            "exception: None\n",
            "\n",
            "19:27:48 job_callback for (2, 0, 0) started\n",
            "19:27:48 DISPATCHER: Trying to submit another job.\n",
            "19:27:48 job_callback for (2, 0, 0) got condition\n",
            "19:27:48 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:48 Only 4 run(s) for budget 1.666667 available, need more than 7 -> can't build model!\n",
            "19:27:48 HBMASTER: Trying to run another job!\n",
            "19:27:48 job_callback for (2, 0, 0) finished\n",
            "19:27:48 start sampling a new configuration.\n",
            "19:27:48 done sampling a new configuration.\n",
            "19:27:48 HBMASTER: schedule new run for iteration 2\n",
            "19:27:48 HBMASTER: trying submitting job (2, 0, 1) to dispatcher\n",
            "19:27:48 HBMASTER: submitting job (2, 0, 1) to dispatcher\n",
            "19:27:48 DISPATCHER: trying to submit job (2, 0, 1)\n",
            "19:27:48 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:48 HBMASTER: job (2, 0, 1) submitted to dispatcher\n",
            "19:27:48 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:48 DISPATCHER: Trying to submit another job.\n",
            "19:27:48 DISPATCHER: starting job (2, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:48 DISPATCHER: job (2, 0, 1) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:48 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:48 WORKER: start processing job (2, 0, 1)\n",
            "19:27:48 WORKER: args: ()\n",
            "19:27:48 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 3, 'learning_rate': 0.14867657349497748, 'units_first': 166, 'units_hidden': 299}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:49 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "22/30 [=====================>........] - ETA: 0s - loss: 0.9587 - accuracy: 0.8920 - Recall: 0.8939 - Precision: 0.8939"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 2s 78ms/step - loss: 0.8560 - accuracy: 0.9073 - Recall: 0.9125 - Precision: 0.9031 - val_loss: 12.7196 - val_accuracy: 0.9154 - val_Recall: 0.9900 - val_Precision: 0.4963\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:51 WORKER: done with job (2, 0, 1), trying to register it.\n",
            "19:27:51 WORKER: registered result for job (2, 0, 1) with dispatcher\n",
            "19:27:51 DISPATCHER: job (2, 0, 1) finished\n",
            "19:27:51 DISPATCHER: register_result: lock acquired\n",
            "19:27:51 DISPATCHER: job (2, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:51 job_id: (2, 0, 1)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 3, 'learning_rate': 0.14867657349497748, 'units_first': 166, 'units_hidden': 299}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.5037406384944916, 'info': {'loss': [0.8559859791087515], 'accuracy': [0.9072916507720947], 'Recall': [0.9125000238418579], 'Precision': [0.9030928015708923], 'val_loss': [12.719560972960494], 'val_accuracy': [0.9153878092765808], 'val_Recall': [0.9900497794151306], 'val_Precision': [0.4962593615055084]}}\n",
            "exception: None\n",
            "\n",
            "19:27:51 job_callback for (2, 0, 1) started\n",
            "19:27:51 DISPATCHER: Trying to submit another job.\n",
            "19:27:51 job_callback for (2, 0, 1) got condition\n",
            "19:27:51 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:51 Only 5 run(s) for budget 1.666667 available, need more than 7 -> can't build model!\n",
            "19:27:51 HBMASTER: Trying to run another job!\n",
            "19:27:51 job_callback for (2, 0, 1) finished\n",
            "19:27:51 start sampling a new configuration.\n",
            "19:27:51 done sampling a new configuration.\n",
            "19:27:51 HBMASTER: schedule new run for iteration 2\n",
            "19:27:51 HBMASTER: trying submitting job (2, 0, 2) to dispatcher\n",
            "19:27:51 HBMASTER: submitting job (2, 0, 2) to dispatcher\n",
            "19:27:51 DISPATCHER: trying to submit job (2, 0, 2)\n",
            "19:27:51 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:51 HBMASTER: job (2, 0, 2) submitted to dispatcher\n",
            "19:27:51 DISPATCHER: Trying to submit another job.\n",
            "19:27:51 DISPATCHER: starting job (2, 0, 2) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:51 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:51 DISPATCHER: job (2, 0, 2) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:51 WORKER: start processing job (2, 0, 2)\n",
            "19:27:51 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:51 WORKER: args: ()\n",
            "19:27:51 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 2, 'learning_rate': 0.08626377975575458, 'units_first': 267, 'units_hidden': 159}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:51 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "23/30 [======================>.......] - ETA: 0s - loss: 0.4708 - accuracy: 0.8913 - Recall: 0.8987 - Precision: 0.8892"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r30/30 [==============================] - 2s 65ms/step - loss: 0.4876 - accuracy: 0.9052 - Recall: 0.9068 - Precision: 0.9050 - val_loss: 2.3431 - val_accuracy: 0.9154 - val_Recall: 0.9851 - val_Precision: 0.4962\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:53 WORKER: done with job (2, 0, 2), trying to register it.\n",
            "19:27:53 WORKER: registered result for job (2, 0, 2) with dispatcher\n",
            "19:27:53 DISPATCHER: job (2, 0, 2) finished\n",
            "19:27:53 DISPATCHER: register_result: lock acquired\n",
            "19:27:53 DISPATCHER: job (2, 0, 2) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:27:53 job_id: (2, 0, 2)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 2, 'learning_rate': 0.08626377975575458, 'units_first': 267, 'units_hidden': 159}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.5037593841552734, 'info': {'loss': [0.48755951643106527], 'accuracy': [0.9052083492279053], 'Recall': [0.9068322777748108], 'Precision': [0.9049586653709412], 'val_loss': [2.3431308444035164], 'val_accuracy': [0.9153878092765808], 'val_Recall': [0.9850746393203735], 'val_Precision': [0.49624061584472656]}}\n",
            "exception: None\n",
            "\n",
            "19:27:53 job_callback for (2, 0, 2) started\n",
            "19:27:53 DISPATCHER: Trying to submit another job.\n",
            "19:27:53 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:27:53 job_callback for (2, 0, 2) got condition\n",
            "19:27:53 HBMASTER: Trying to run another job!\n",
            "19:27:53 job_callback for (2, 0, 2) finished\n",
            "19:27:53 ITERATION: Advancing config (2, 0, 0) to next budget 5.000000\n",
            "19:27:53 HBMASTER: schedule new run for iteration 2\n",
            "19:27:53 HBMASTER: trying submitting job (2, 0, 0) to dispatcher\n",
            "19:27:53 HBMASTER: submitting job (2, 0, 0) to dispatcher\n",
            "19:27:53 DISPATCHER: trying to submit job (2, 0, 0)\n",
            "19:27:53 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:27:53 HBMASTER: job (2, 0, 0) submitted to dispatcher\n",
            "19:27:53 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:27:53 DISPATCHER: Trying to submit another job.\n",
            "19:27:54 DISPATCHER: starting job (2, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:54 DISPATCHER: job (2, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:27:54 WORKER: start processing job (2, 0, 0)\n",
            "19:27:54 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:27:54 WORKER: args: ()\n",
            "19:27:54 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 4, 'learning_rate': 0.09400132689549366, 'units_first': 227, 'units_hidden': 65}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:27:54 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "Epoch 1/5\n",
            "13/30 [============>.................] - ETA: 2s - loss: 0.3373 - accuracy: 0.8606 - Recall: 0.8224 - Precision: 0.8980"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 86ms/step - loss: 0.2161 - accuracy: 0.9240 - Recall: 0.8954 - Precision: 0.9490 - val_loss: 5.6169 - val_accuracy: 0.8909 - val_Recall: 0.9900 - val_Precision: 0.4326\n",
            "Epoch 2/5\n",
            "30/30 [==============================] - 1s 20ms/step - loss: 0.1241 - accuracy: 0.9718 - Recall: 0.9707 - Precision: 0.9727 - val_loss: 1.0447 - val_accuracy: 0.9361 - val_Recall: 0.9851 - val_Precision: 0.5673\n",
            "Epoch 3/5\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.1135 - accuracy: 0.9729 - Recall: 0.9725 - Precision: 0.9725 - val_loss: 0.4816 - val_accuracy: 0.9548 - val_Recall: 0.9851 - val_Precision: 0.6513\n",
            "Epoch 4/5\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.0889 - accuracy: 0.9791 - Recall: 0.9748 - Precision: 0.9831 - val_loss: 0.3233 - val_accuracy: 0.9672 - val_Recall: 0.9851 - val_Precision: 0.7226\n",
            "Epoch 5/5\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.0805 - accuracy: 0.9760 - Recall: 0.9710 - Precision: 0.9811 - val_loss: 0.2543 - val_accuracy: 0.9743 - val_Recall: 0.9801 - val_Precision: 0.7725\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:00 WORKER: done with job (2, 0, 0), trying to register it.\n",
            "19:28:00 WORKER: registered result for job (2, 0, 0) with dispatcher\n",
            "19:28:00 DISPATCHER: job (2, 0, 0) finished\n",
            "19:28:00 DISPATCHER: register_result: lock acquired\n",
            "19:28:00 DISPATCHER: job (2, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:00 job_id: (2, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 4, 'learning_rate': 0.09400132689549366, 'units_first': 227, 'units_hidden': 65}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.22745096683502197, 'info': {'loss': [0.21606949440514048, 0.12429978323629853, 0.11368249904652104, 0.0889950733484082, 0.08066869574056507], 'accuracy': [0.9239583611488342, 0.9718163013458252, 0.9728600978851318, 0.9791231751441956, 0.9759916663169861], 'Recall': [0.8953974843025208, 0.9707112908363342, 0.972515881061554, 0.9748427867889404, 0.9709543585777283], 'Precision': [0.949002206325531, 0.9727463126182556, 0.972515881061554, 0.9830867052078247, 0.9811320900917053], 'val_loss': [5.616881680049454, 1.0446641991140462, 0.48155227069889667, 0.3232717369502018, 0.25427709164475304], 'val_accuracy': [0.890916645526886, 0.9361261129379272, 0.9547905325889587, 0.9672335386276245, 0.9742845296859741], 'val_Recall': [0.9900497794151306, 0.9850746393203735, 0.9850746393203735, 0.9850746393203735, 0.9800994992256165], 'val_Precision': [0.4326086938381195, 0.5673352479934692, 0.6513158082962036, 0.7226277589797974, 0.772549033164978]}}\n",
            "exception: None\n",
            "\n",
            "19:28:00 job_callback for (2, 0, 0) started\n",
            "19:28:00 DISPATCHER: Trying to submit another job.\n",
            "19:28:00 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:00 job_callback for (2, 0, 0) got condition\n",
            "19:28:00 Only 4 run(s) for budget 5.000000 available, need more than 7 -> can't build model!\n",
            "19:28:00 HBMASTER: Trying to run another job!\n",
            "19:28:00 job_callback for (2, 0, 0) finished\n",
            "19:28:00 start sampling a new configuration.\n",
            "19:28:00 done sampling a new configuration.\n",
            "19:28:00 HBMASTER: schedule new run for iteration 3\n",
            "19:28:00 HBMASTER: trying submitting job (3, 0, 0) to dispatcher\n",
            "19:28:00 HBMASTER: submitting job (3, 0, 0) to dispatcher\n",
            "19:28:00 DISPATCHER: trying to submit job (3, 0, 0)\n",
            "19:28:00 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:28:00 HBMASTER: job (3, 0, 0) submitted to dispatcher\n",
            "19:28:00 DISPATCHER: Trying to submit another job.\n",
            "19:28:00 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:28:00 DISPATCHER: starting job (3, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:00 DISPATCHER: job (3, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:00 WORKER: start processing job (3, 0, 0)\n",
            "19:28:00 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:28:00 WORKER: args: ()\n",
            "19:28:00 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 3, 'learning_rate': 0.11172259364617185, 'units_first': 157, 'units_hidden': 76}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:00 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "Epoch 1/5\n",
            "22/30 [=====================>........] - ETA: 0s - loss: 0.2748 - accuracy: 0.9062 - Recall: 0.8901 - Precision: 0.9257"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 2s 77ms/step - loss: 0.2488 - accuracy: 0.9219 - Recall: 0.9040 - Precision: 0.9372 - val_loss: 2.5695 - val_accuracy: 0.8785 - val_Recall: 0.9950 - val_Precision: 0.4065\n",
            "Epoch 2/5\n",
            "30/30 [==============================] - 1s 19ms/step - loss: 0.1791 - accuracy: 0.9593 - Recall: 0.9542 - Precision: 0.9642 - val_loss: 0.5036 - val_accuracy: 0.9515 - val_Recall: 0.9851 - val_Precision: 0.6346\n",
            "Epoch 3/5\n",
            "30/30 [==============================] - 1s 19ms/step - loss: 0.1593 - accuracy: 0.9572 - Recall: 0.9564 - Precision: 0.9584 - val_loss: 0.2346 - val_accuracy: 0.9710 - val_Recall: 0.9851 - val_Precision: 0.7472\n",
            "Epoch 4/5\n",
            "30/30 [==============================] - 1s 19ms/step - loss: 0.1181 - accuracy: 0.9656 - Recall: 0.9664 - Precision: 0.9644 - val_loss: 0.1680 - val_accuracy: 0.9830 - val_Recall: 0.9801 - val_Precision: 0.8419\n",
            "Epoch 5/5\n",
            "30/30 [==============================] - 1s 19ms/step - loss: 0.1487 - accuracy: 0.9582 - Recall: 0.9563 - Precision: 0.9603 - val_loss: 0.1035 - val_accuracy: 0.9813 - val_Recall: 0.9652 - val_Precision: 0.8362\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:05 WORKER: done with job (3, 0, 0), trying to register it.\n",
            "19:28:05 WORKER: registered result for job (3, 0, 0) with dispatcher\n",
            "19:28:05 DISPATCHER: job (3, 0, 0) finished\n",
            "19:28:05 DISPATCHER: register_result: lock acquired\n",
            "19:28:05 DISPATCHER: job (3, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:05 job_id: (3, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 3, 'learning_rate': 0.11172259364617185, 'units_first': 157, 'units_hidden': 76}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.16379308700561523, 'info': {'loss': [0.248823786744227, 0.17942454831554147, 0.15955787834892193, 0.11826853282819648, 0.148943183630596], 'accuracy': [0.921875, 0.9592902064323425, 0.9572024941444397, 0.9655532240867615, 0.9582463502883911], 'Recall': [0.9039666056632996, 0.9541666507720947, 0.95643150806427, 0.9663865566253662, 0.9563409686088562], 'Precision': [0.9372294545173645, 0.9642105102539062, 0.9584199786186218, 0.9643605947494507, 0.9603340029716492], 'val_loss': [2.5695286071483583, 0.5035667460423158, 0.23455136941145405, 0.16802552925278813, 0.10345224627161805], 'val_accuracy': [0.8784736394882202, 0.9514724016189575, 0.9709663987159729, 0.9829946160316467, 0.9813355207443237], 'val_Recall': [0.9950248599052429, 0.9850746393203735, 0.9850746393203735, 0.9800994992256165, 0.96517413854599], 'val_Precision': [0.4065040647983551, 0.6346153616905212, 0.7471697926521301, 0.8418803215026855, 0.8362069129943848]}}\n",
            "exception: None\n",
            "\n",
            "19:28:05 job_callback for (3, 0, 0) started\n",
            "19:28:05 DISPATCHER: Trying to submit another job.\n",
            "19:28:05 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:05 job_callback for (3, 0, 0) got condition\n",
            "19:28:05 Only 5 run(s) for budget 5.000000 available, need more than 7 -> can't build model!\n",
            "19:28:05 HBMASTER: Trying to run another job!\n",
            "19:28:05 job_callback for (3, 0, 0) finished\n",
            "19:28:05 start sampling a new configuration.\n",
            "19:28:05 done sampling a new configuration.\n",
            "19:28:05 HBMASTER: schedule new run for iteration 3\n",
            "19:28:05 HBMASTER: trying submitting job (3, 0, 1) to dispatcher\n",
            "19:28:05 HBMASTER: submitting job (3, 0, 1) to dispatcher\n",
            "19:28:05 DISPATCHER: trying to submit job (3, 0, 1)\n",
            "19:28:05 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:28:05 HBMASTER: job (3, 0, 1) submitted to dispatcher\n",
            "19:28:05 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:28:05 DISPATCHER: Trying to submit another job.\n",
            "19:28:05 DISPATCHER: starting job (3, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:05 DISPATCHER: job (3, 0, 1) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:05 WORKER: start processing job (3, 0, 1)\n",
            "19:28:05 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:28:05 WORKER: args: ()\n",
            "19:28:05 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.033841489580347335, 'units_first': 59, 'units_hidden': 74}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:05 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "Epoch 1/5\n",
            "12/30 [===========>..................] - ETA: 3s - loss: 0.2995 - accuracy: 0.8802 - Recall: 0.8756 - Precision: 0.8934"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 109ms/step - loss: 0.1920 - accuracy: 0.9312 - Recall: 0.9245 - Precision: 0.9363 - val_loss: 6.7048 - val_accuracy: 0.8818 - val_Recall: 0.9950 - val_Precision: 0.4132\n",
            "Epoch 2/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.0939 - accuracy: 0.9645 - Recall: 0.9667 - Precision: 0.9627 - val_loss: 1.1259 - val_accuracy: 0.9320 - val_Recall: 0.9851 - val_Precision: 0.5515\n",
            "Epoch 3/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.0881 - accuracy: 0.9718 - Recall: 0.9686 - Precision: 0.9747 - val_loss: 0.6148 - val_accuracy: 0.9552 - val_Recall: 0.9851 - val_Precision: 0.6535\n",
            "Epoch 4/5\n",
            "30/30 [==============================] - 1s 23ms/step - loss: 0.0653 - accuracy: 0.9864 - Recall: 0.9811 - Precision: 0.9915 - val_loss: 0.2530 - val_accuracy: 0.9759 - val_Recall: 0.9851 - val_Precision: 0.7826\n",
            "Epoch 5/5\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.0477 - accuracy: 0.9875 - Recall: 0.9937 - Precision: 0.9814 - val_loss: 0.2196 - val_accuracy: 0.9797 - val_Recall: 0.9801 - val_Precision: 0.8140\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:11 WORKER: done with job (3, 0, 1), trying to register it.\n",
            "19:28:11 WORKER: registered result for job (3, 0, 1) with dispatcher\n",
            "19:28:11 DISPATCHER: job (3, 0, 1) finished\n",
            "19:28:11 DISPATCHER: register_result: lock acquired\n",
            "19:28:11 DISPATCHER: job (3, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:11 job_id: (3, 0, 1)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.033841489580347335, 'units_first': 59, 'units_hidden': 74}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.1859503984451294, 'info': {'loss': [0.19195289687874417, 0.09387230946213816, 0.08813948136810719, 0.06538860609906576, 0.047743067864740814], 'accuracy': [0.9312499761581421, 0.9645093679428101, 0.9718163013458252, 0.9864300489425659, 0.9874739050865173], 'Recall': [0.9245283007621765, 0.9666666388511658, 0.9686192274093628, 0.9811320900917053, 0.9937369227409363], 'Precision': [0.9363057613372803, 0.9626556038856506, 0.9747368693351746, 0.991525411605835, 0.9814432859420776], 'val_loss': [6.704797026519982, 1.125928996977896, 0.6148153617761284, 0.2530497998580751, 0.21956406923651475], 'val_accuracy': [0.8817917704582214, 0.9319784045219421, 0.9552053213119507, 0.9759435653686523, 0.9796764850616455], 'val_Recall': [0.9950248599052429, 0.9850746393203735, 0.9850746393203735, 0.9850746393203735, 0.9800994992256165], 'val_Precision': [0.41322314739227295, 0.5515320301055908, 0.6534653306007385, 0.782608687877655, 0.8140496015548706]}}\n",
            "exception: None\n",
            "\n",
            "19:28:11 job_callback for (3, 0, 1) started\n",
            "19:28:11 DISPATCHER: Trying to submit another job.\n",
            "19:28:11 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:11 job_callback for (3, 0, 1) got condition\n",
            "19:28:11 HBMASTER: Trying to run another job!\n",
            "19:28:11 job_callback for (3, 0, 1) finished\n",
            "19:28:11 start sampling a new configuration.\n",
            "19:28:11 done sampling a new configuration.\n",
            "19:28:11 HBMASTER: schedule new run for iteration 4\n",
            "19:28:11 HBMASTER: trying submitting job (4, 0, 0) to dispatcher\n",
            "19:28:11 HBMASTER: submitting job (4, 0, 0) to dispatcher\n",
            "19:28:11 DISPATCHER: trying to submit job (4, 0, 0)\n",
            "19:28:11 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:28:11 HBMASTER: job (4, 0, 0) submitted to dispatcher\n",
            "19:28:11 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:28:11 DISPATCHER: Trying to submit another job.\n",
            "19:28:11 DISPATCHER: starting job (4, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:11 DISPATCHER: job (4, 0, 0) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:11 WORKER: start processing job (4, 0, 0)\n",
            "19:28:11 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:28:11 WORKER: args: ()\n",
            "19:28:11 WORKER: kwargs: {'config': {'batch_size': 64, 'dense_layers': 4, 'learning_rate': 0.08034127476223765, 'units_first': 3, 'units_hidden': 99}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:12 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 15 steps, validate on 2411 samples\n",
            "12/15 [=======================>......] - ETA: 0s - loss: 0.7159 - accuracy: 0.7044 - Recall: 0.6718 - Precision: 0.7258"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r15/15 [==============================] - 2s 155ms/step - loss: 0.6468 - accuracy: 0.7344 - Recall: 0.6998 - Precision: 0.7545 - val_loss: 15.8352 - val_accuracy: 0.5218 - val_Recall: 1.0000 - val_Precision: 0.1484\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:14 WORKER: done with job (4, 0, 0), trying to register it.\n",
            "19:28:14 WORKER: registered result for job (4, 0, 0) with dispatcher\n",
            "19:28:14 DISPATCHER: job (4, 0, 0) finished\n",
            "19:28:14 DISPATCHER: register_result: lock acquired\n",
            "19:28:14 DISPATCHER: job (4, 0, 0) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:14 job_id: (4, 0, 0)\n",
            "kwargs: {'config': {'batch_size': 64, 'dense_layers': 4, 'learning_rate': 0.08034127476223765, 'units_first': 3, 'units_hidden': 99}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.851550966501236, 'info': {'loss': [0.6468040237824122], 'accuracy': [0.734375], 'Recall': [0.6997929811477661], 'Precision': [0.7544642686843872], 'val_loss': [15.83516667325286], 'val_accuracy': [0.5217751860618591], 'val_Recall': [1.0], 'val_Precision': [0.14844903349876404]}}\n",
            "exception: None\n",
            "\n",
            "19:28:14 job_callback for (4, 0, 0) started\n",
            "19:28:14 DISPATCHER: Trying to submit another job.\n",
            "19:28:14 job_callback for (4, 0, 0) got condition\n",
            "19:28:14 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:14 HBMASTER: Trying to run another job!\n",
            "19:28:14 job_callback for (4, 0, 0) finished\n",
            "19:28:14 start sampling a new configuration.\n",
            "19:28:14 done sampling a new configuration.\n",
            "19:28:14 HBMASTER: schedule new run for iteration 4\n",
            "19:28:14 HBMASTER: trying submitting job (4, 0, 1) to dispatcher\n",
            "19:28:14 HBMASTER: submitting job (4, 0, 1) to dispatcher\n",
            "19:28:14 DISPATCHER: trying to submit job (4, 0, 1)\n",
            "19:28:14 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:28:14 HBMASTER: job (4, 0, 1) submitted to dispatcher\n",
            "19:28:14 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:28:14 DISPATCHER: Trying to submit another job.\n",
            "19:28:14 DISPATCHER: starting job (4, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:14 DISPATCHER: job (4, 0, 1) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:14 WORKER: start processing job (4, 0, 1)\n",
            "19:28:14 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:28:14 WORKER: args: ()\n",
            "19:28:14 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.09772164772724096, 'units_first': 282, 'units_hidden': 17}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:15 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "11/30 [==========>...................] - ETA: 3s - loss: 0.4747 - accuracy: 0.7784 - Recall: 0.7778 - Precision: 0.8033"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 95ms/step - loss: 0.2968 - accuracy: 0.8802 - Recall: 0.8776 - Precision: 0.8831 - val_loss: 16.4727 - val_accuracy: 0.8905 - val_Recall: 0.9900 - val_Precision: 0.4317\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:18 WORKER: done with job (4, 0, 1), trying to register it.\n",
            "19:28:18 WORKER: registered result for job (4, 0, 1) with dispatcher\n",
            "19:28:18 DISPATCHER: job (4, 0, 1) finished\n",
            "19:28:18 DISPATCHER: register_result: lock acquired\n",
            "19:28:18 DISPATCHER: job (4, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:18 job_id: (4, 0, 1)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.09772164772724096, 'units_first': 282, 'units_hidden': 17}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.5683297216892242, 'info': {'loss': [0.29680303496619065], 'accuracy': [0.8802083134651184], 'Recall': [0.8775933384895325], 'Precision': [0.8830897808074951], 'val_loss': [16.472664682400737], 'val_accuracy': [0.890501856803894], 'val_Recall': [0.9900497794151306], 'val_Precision': [0.43167027831077576]}}\n",
            "exception: None\n",
            "\n",
            "19:28:18 job_callback for (4, 0, 1) started\n",
            "19:28:18 job_callback for (4, 0, 1) got condition\n",
            "19:28:18 DISPATCHER: Trying to submit another job.\n",
            "19:28:18 HBMASTER: Trying to run another job!\n",
            "19:28:18 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:18 job_callback for (4, 0, 1) finished\n",
            "19:28:18 start sampling a new configuration.\n",
            "19:28:18 done sampling a new configuration.\n",
            "19:28:18 HBMASTER: schedule new run for iteration 4\n",
            "19:28:18 HBMASTER: trying submitting job (4, 0, 2) to dispatcher\n",
            "19:28:18 HBMASTER: submitting job (4, 0, 2) to dispatcher\n",
            "19:28:18 DISPATCHER: trying to submit job (4, 0, 2)\n",
            "19:28:18 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:28:18 HBMASTER: job (4, 0, 2) submitted to dispatcher\n",
            "19:28:18 DISPATCHER: Trying to submit another job.\n",
            "19:28:18 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:28:18 DISPATCHER: starting job (4, 0, 2) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:18 DISPATCHER: job (4, 0, 2) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:18 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:28:18 WORKER: start processing job (4, 0, 2)\n",
            "19:28:18 WORKER: args: ()\n",
            "19:28:18 WORKER: kwargs: {'config': {'batch_size': 64, 'dense_layers': 2, 'learning_rate': 0.018794922767736623, 'units_first': 272, 'units_hidden': 171}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:18 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 15 steps, validate on 2411 samples\n",
            " 8/15 [===============>..............] - ETA: 1s - loss: 0.2515 - accuracy: 0.9102 - Recall: 0.8960 - Precision: 0.9180 "
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r15/15 [==============================] - 2s 150ms/step - loss: 0.2028 - accuracy: 0.9385 - Recall: 0.9354 - Precision: 0.9413 - val_loss: 1.7174 - val_accuracy: 0.8109 - val_Recall: 1.0000 - val_Precision: 0.3059\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:20 WORKER: done with job (4, 0, 2), trying to register it.\n",
            "19:28:20 WORKER: registered result for job (4, 0, 2) with dispatcher\n",
            "19:28:20 DISPATCHER: job (4, 0, 2) finished\n",
            "19:28:20 DISPATCHER: register_result: lock acquired\n",
            "19:28:20 DISPATCHER: job (4, 0, 2) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:20 job_id: (4, 0, 2)\n",
            "kwargs: {'config': {'batch_size': 64, 'dense_layers': 2, 'learning_rate': 0.018794922767736623, 'units_first': 272, 'units_hidden': 171}, 'budget': 1.6666666666666665, 'working_directory': '.'}\n",
            "result: {'loss': 0.6940639317035675, 'info': {'loss': [0.20284775644540787], 'accuracy': [0.9385416507720947], 'Recall': [0.9354166388511658], 'Precision': [0.9412997961044312], 'val_loss': [1.7173542313830876], 'val_accuracy': [0.8108668327331543], 'val_Recall': [1.0], 'val_Precision': [0.3059360682964325]}}\n",
            "exception: None\n",
            "\n",
            "19:28:20 job_callback for (4, 0, 2) started\n",
            "19:28:20 DISPATCHER: Trying to submit another job.\n",
            "19:28:20 job_callback for (4, 0, 2) got condition\n",
            "19:28:20 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:20 HBMASTER: Trying to run another job!\n",
            "19:28:20 job_callback for (4, 0, 2) finished\n",
            "19:28:20 ITERATION: Advancing config (4, 0, 1) to next budget 5.000000\n",
            "19:28:20 HBMASTER: schedule new run for iteration 4\n",
            "19:28:20 HBMASTER: trying submitting job (4, 0, 1) to dispatcher\n",
            "19:28:20 HBMASTER: submitting job (4, 0, 1) to dispatcher\n",
            "19:28:20 DISPATCHER: trying to submit job (4, 0, 1)\n",
            "19:28:20 DISPATCHER: trying to notify the job_runner thread.\n",
            "19:28:20 HBMASTER: job (4, 0, 1) submitted to dispatcher\n",
            "19:28:20 HBMASTER: running jobs: 1, queue sizes: (0, 1) -> wait\n",
            "19:28:20 DISPATCHER: Trying to submit another job.\n",
            "19:28:20 DISPATCHER: starting job (4, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:20 DISPATCHER: job (4, 0, 1) dispatched on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184\n",
            "19:28:20 DISPATCHER: jobs to submit = 0, number of idle workers = 0 -> waiting!\n",
            "19:28:20 WORKER: start processing job (4, 0, 1)\n",
            "19:28:20 WORKER: args: ()\n",
            "19:28:20 WORKER: kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.09772164772724096, 'units_first': 282, 'units_hidden': 17}, 'budget': 5.0, 'working_directory': '.'}\n",
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:21 sample_weight modes were coerced from\n",
            "  ...\n",
            "    to  \n",
            "  ['...']\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train for 30 steps, validate on 2411 samples\n",
            "Epoch 1/5\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:22 DISPATCHER: Starting worker discovery\n",
            "19:28:22 DISPATCHER: Found 1 potential workers, 1 currently in the pool.\n",
            "19:28:22 DISPATCHER: Finished worker discovery\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "12/30 [===========>..................] - ETA: 3s - loss: 0.5060 - accuracy: 0.7474 - Recall: 0.7268 - Precision: 0.7622"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function safe_indexing is deprecated; safe_indexing is deprecated in version 0.22 and will be removed in version 0.24.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "30/30 [==============================] - 3s 98ms/step - loss: 0.3466 - accuracy: 0.8573 - Recall: 0.8372 - Precision: 0.8717 - val_loss: 4.2293 - val_accuracy: 0.8698 - val_Recall: 0.9950 - val_Precision: 0.3899\n",
            "Epoch 2/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.2081 - accuracy: 0.9405 - Recall: 0.9314 - Precision: 0.9492 - val_loss: 0.5620 - val_accuracy: 0.9486 - val_Recall: 0.9851 - val_Precision: 0.6207\n",
            "Epoch 3/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.1775 - accuracy: 0.9562 - Recall: 0.9623 - Precision: 0.9504 - val_loss: 0.1840 - val_accuracy: 0.9768 - val_Recall: 0.9851 - val_Precision: 0.7888\n",
            "Epoch 4/5\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.1441 - accuracy: 0.9562 - Recall: 0.9582 - Precision: 0.9543 - val_loss: 0.1806 - val_accuracy: 0.9818 - val_Recall: 0.9851 - val_Precision: 0.8285\n",
            "Epoch 5/5\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.1315 - accuracy: 0.9551 - Recall: 0.9605 - Precision: 0.9506 - val_loss: 0.2440 - val_accuracy: 0.9784 - val_Recall: 0.9851 - val_Precision: 0.8016\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "19:28:27 WORKER: done with job (4, 0, 1), trying to register it.\n",
            "19:28:27 WORKER: registered result for job (4, 0, 1) with dispatcher\n",
            "19:28:27 DISPATCHER: job (4, 0, 1) finished\n",
            "19:28:27 DISPATCHER: register_result: lock acquired\n",
            "19:28:27 DISPATCHER: job (4, 0, 1) on hpbandster.run_MLP.worker.6cc0892967ab.875139915884685184 finished\n",
            "19:28:27 job_id: (4, 0, 1)\n",
            "kwargs: {'config': {'batch_size': 32, 'dense_layers': 5, 'learning_rate': 0.09772164772724096, 'units_first': 282, 'units_hidden': 17}, 'budget': 5.0, 'working_directory': '.'}\n",
            "result: {'loss': 0.19838058948516846, 'info': {'loss': [0.3465707225104173, 0.2083690301019364, 0.177652965025638, 0.14422870043175207, 0.1315531540265412], 'accuracy': [0.8572916388511658, 0.9405010342597961, 0.9561586380004883, 0.9561586380004883, 0.9551148414611816], 'Recall': [0.8371607661247253, 0.9313929080963135, 0.9623430967330933, 0.9582463502883911, 0.9604989886283875], 'Precision': [0.8717391490936279, 0.9491525292396545, 0.9504132270812988, 0.9542619585990906, 0.9506173133850098], 'val_loss': [4.229287137737701, 0.5620401551240656, 0.18399557053200288, 0.18063165117280322, 0.24398054908297037], 'val_accuracy': [0.8697636127471924, 0.9485690593719482, 0.9767731428146362, 0.9817503094673157, 0.9784321784973145], 'val_Recall': [0.9950248599052429, 0.9850746393203735, 0.9850746393203735, 0.9850746393203735, 0.9850746393203735], 'val_Precision': [0.3898635506629944, 0.6206896305084229, 0.788844645023346, 0.8284518718719482, 0.8016194105148315]}}\n",
            "exception: None\n",
            "\n",
            "19:28:27 job_callback for (4, 0, 1) started\n",
            "19:28:27 DISPATCHER: Trying to submit another job.\n",
            "19:28:27 job_callback for (4, 0, 1) got condition\n",
            "19:28:27 DISPATCHER: jobs to submit = 0, number of idle workers = 1 -> waiting!\n",
            "19:28:27 HBMASTER: Trying to run another job!\n",
            "19:28:27 job_callback for (4, 0, 1) finished\n",
            "19:28:27 HBMASTER: shutdown initiated, shutdown_workers = True\n",
            "19:28:27 WORKER: shutting down now!\n",
            "19:28:27 DISPATCHER: Dispatcher shutting down\n",
            "19:28:27 DISPATCHER: Trying to submit another job.\n",
            "19:28:27 DISPATCHER: job_runner shutting down\n",
            "19:28:27 DISPATCHER: discover_workers shutting down\n",
            "19:28:27 DISPATCHER: 'discover_worker' thread exited\n",
            "19:28:27 DISPATCHER: 'job_runner' thread exited\n",
            "19:28:27 DISPATCHER: shut down complete\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8CM6uv09dJ4B",
        "colab_type": "text"
      },
      "source": [
        "**TODO**\n",
        "- add the visualisations I made from the GPU paper"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b61pfegje_Ok",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XG2zc0pJ3L9Y",
        "colab_type": "text"
      },
      "source": [
        "# Exercises\n",
        "\n",
        "Below are a few suggested exercises that may help improve your skills.\n",
        "\n",
        "You may want to backup this Notebook before making any changes.\n",
        "\n",
        "1. get the HpBandSter model to minimise F1-score rather than precision"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v1mW9gbA3Saj",
        "colab_type": "text"
      },
      "source": [
        "# License\n",
        "\n",
        "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br />This work is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>."
      ]
    }
  ]
}